ETL Pipelines

🔄 ETL Pipelines

Data Transformation, Pipeline Orchestration, Data Integration, Workflow Management

Feeds to Scour
SubscribedAll
Scoured 116 posts in 10.2 ms

COSS Weekly: Supabase achieves $10B valuation, DeepSeek eyes $7B funding round, Martin Scorsese joins Black Forest Labs, and more

 🔧Data Engineering  Content type: Blog
dev.to··DEV

Real-time CDC from Aurora PostgreSQL to Amazon S3 Tables using Debezium and Firehose

 📊AWS Analytics  Content type: Blog
aws.amazon.com·

Senior Data Engineer – Climate Friendly

 🔧Data Engineering
au.seek.com··Hacker News, Hacker News

Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04

 🚀DevOps  Content type: Reference  Content type: Tutorial
docs.vultr.com··DEV

Why the Modern Data Stack Trapped Data Engineers in Tools

 🏗️Data Platforms
hackernoon.com·

Storage news ticker - 4 June

 🔧Data Engineering
blocksandfiles.com·

Real-time data replication to your data warehouse, self-serve

 🏗️Data Platforms
artie.com··Hacker News, Hacker News

Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance

 🌟spark  Content type: Blog
cloud.google.com·

Modern Data Stack Migration — Day 1: Scaling to 8+ Companies with DRY Architecture and Chasing a $2M Discrepancy

 🏗️Data Platforms  Content type: Blog
dev.to··DEV

benseverndev-oss/goldenmatch: Zero-config entity resolution that scales from a CSV to 100M+ rows on a Ray cluster (verified: 100M deduped in 213s, 0.30 GB driver). Fuzzy + exact + probabilistic dedupe, identity graph, PPRL, LLM boost. Python + full TypeScript port; SQL-native in PostgreSQL & DuckDB; MCP/REST servers, dbt + Airflow recipes.

 💾Databases  Content type: Code
github.com··Hacker News

DuckDB Storage Engine for MariaDB. When the Sea Lion Learns to Quack.

 📊Columnar Engines
mariadb.org··Hacker News

ETLs in the Era of AI and Sandboxes

 📋CSV Processing
zozo123.github.io··Hacker News

Turso libSQL vs Cloudflare D1 for an Astro monorepo: the practical difference

 💾Databases  Content type: Blog
dev.to··DEV

Paramount is merging Pluto TV, BET+, and Paramount+ onto one tech stack to prepare for HBO Max

 🔧Data Engineering  Content type: News
thenextweb.com·

Gene dependency-informed inference of response to targeted cancer therapies

 🔧Data Engineering  Content type: Academic
nature.com·

Your RAG System Might Be Confidently Wrong

 🔧Data Engineering
hackernoon.com·

Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions

 ☁️Cloud Computing  Content type: Blog
aws.amazon.com·

How I built a three-tier content quality ladder for programmatic directory ETL

 📋CSV Processing
platform.claude.com··DEV

Day 10 of 100 Days of ClickHouse®: What Makes ClickHouse SQL Different?

 📊Column Stores
quantrail-data.com··DEV

From Data Quality Checks to Analytics-Ready Parquet with Python

 📋CSV Processing  Content type: Blog
dev.to··DEV

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help