🏗 data engineering - widget101 · Scour

PySpark : The Big Brain of Data Processing 🌟spark

dev.to·2d·DEV·

Agent Skills: One-Shot PySpark from the CLI 🌟spark

lakesail.com·6d·Hacker News·

lynxbase/lynxdb: A lightweight schema-on-read analytics in a single binary ⚡DataFusion

github.com·2d·Hacker News·

What Category Theory Teaches Us About DataFrames 🧊Iceberg Tables

mchav.github.io·4d·Lobsters, Hacker News, Hacker News, r/programming·

From Pipelines to AI Platforms: How Agentic AI Is Redefining the Role of Data Engineers 🔍AI Detection

hackernoon.com·2d·

Semlib: Semantic Data Processing 🧮Apache Calcite

anishathalye.com·2d·Hacker News·

GA4 Data Quality Monitoring with BigQuery SQL ⏱️Real-time Analytics

paolobietolini.com·4d·

Getting Data from Multiple Sources in Power BI: A Practical Guide to Modern Data Integration for Analysts 📋CSV Processing

yourcompany.sharepoint.com·2d·DEV·

Show HN: Built Loony for builders who want to spin up data infrastructure fast 🧮Apache Calcite

loony.dev·6d·Hacker News·

Drizby: An Open Source BI Platform Built on a Semantic Layer (and why I built it) 🧮Apache Calcite

dev.to·9h·DEV·

Dux: Distributed DuckDB-native DataFrames for Elixir 🌟spark

dux.now·5d·Hacker News·

Build AWS Glue Data Quality pipeline using Terraform 🔗AWS Glue

aws.amazon.com·6d·

From SQL Analytics to Predictive Decision Systems: Operationalizing ML Models in Business Operation ⏱️Real-time Analytics

hackernoon.com·2d·

grove/pg-trickle: A PostgreSQL extension for streaming tables with incremental view maintenance, powered by differential dataflow in Rust. 🧊Iceberg Tables

github.com·2d·Hacker News·

DataFuse.Net - Data Integration Framework. ⚡DataFusion

dev.to·2d·DEV·

Show HN: Diffly – A Python package to compare polars dataframes 🐻Polars

github.com·2d·Hacker News·

How to Optimize Big Data Platform Costs Across the Data Lifecycle 🗄️Storage Tiering

hackernoon.com·3d·

AWS Vector Databases – Part 3 : Choosing the Right Vector Database on AWS ⚡ClickHouse

dev.to·2d·DEV·

pg-warehouse - A local-first data warehouse at scale without over Engineering that mirrors PostgreSQL data 🏛️Lakehouse Architecture

dev.to·2d·DEV·

Building AI Agents That Close the Loop on Pipeline Failures 🤖Copilot

hackernoon.com·3d·

Loading more...