ETL Pipelines

🔄 ETL Pipelines

Data Engineering, Apache Airflow, Pipeline Orchestration, Data Transformation

Feeds to Scour
SubscribedAll
Scoured 125 posts in 9.5 ms

Run an Apache Airflow DAG with Docker Compose and PostgreSQL

 🔧Data Engineering
pyimagesearch.com·

SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development

 🔧Data Engineering  Content type: Blog
medium.com·

Daily Deal: The 2026 Data Engineering Bundle featuring Databricks

 🦆DuckDB
techdirt.com·

Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices

 🔌Data Integration  Content type: Blog
medium.com
·

Linux Fundamentals for Data Engineering

 🦆DuckDB
dev-to-uploads.s3.amazonaws.com··DEV

Senior Data Engineer – Climate Friendly

 🔌Data Integration
au.seek.com··Hacker News, Hacker News

Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions

 🔧Data Engineering  Content type: Blog
aws.amazon.com·

Introducing Flights: Agent-Native Ingest in MotherDuck

 🔌Data Integration  Content type: Blog
motherduck.com·

Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service

 🔌Data Integration
siliconangle.com·

Embedding pipelines are the new ETL

 🔌Data Integration  Content type: Blog
infoworld.com·

Introducing Streamling: Performant and Extensible Data Streaming Framework

 ⚙️DataFusion  Content type: News
streamingdata.tech·

CSU Student of Distinction: Anthony Arthur

 🔧Data Engineering  Content type: Academic
csuohio.edu·

Microsoft just shared the frontier data engineering secrets

 🦆DuckDB
mail.bycloud.ai·

benseverndev-oss/goldenmatch: Zero-config entity resolution that scales from a CSV to 100M+ rows on a Ray cluster (verified: 100M deduped in 213s, 0.30 GB driver). Fuzzy + exact + probabilistic dedupe, identity graph, PPRL, LLM boost. Python + full TypeScript port; SQL-native in PostgreSQL & DuckDB; MCP/REST servers, dbt + Airflow recipes.

 🐍Python  Content type: Code
github.com··Hacker News

Integration Patterns: How To Choose for Your Architecture

 🔌Data Integration  Content type: Blog
blog.n8n.io·

New comment by aldoakhanov in "Ask HN: Who wants to be hired? (June 2026)"

 🐍Python
castlefootyai.com··Hacker News

Central Bank strengthens data governance for AI solutions

 📚Data Catalogs  Content type: News
en.apa.az·

Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance

 ⚡Apache Spark  Content type: Blog
cloud.google.com·

When Feature Importance Lies: Target Encoding at the Noise Floor

 🦆DuckDB
flyback.ai··DEV

New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"

 🐍Python  Content type: PDF
markokolarek.com··Hacker News

Keyboard Shortcuts

Navigation

Next / previous item
j/k
Open post
oorEnter
Preview post
v

Post Actions

Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s

Recommendations

Add interest / feed
Enter
Not interested
x

Go to

Home
gh
Interests
gi
Feeds
gf
Likes
gl
History
gy
Changelog
gc
Settings
gs
Browse
gb
Search
/

General

Show this help
?
Submit feedback
!
Close modal / unfocus
Esc

Press ? anytime to show this help