Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Pipelines
🔄 Data Pipelines
ETL, stream processing, batch processing, data engineering
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
189
posts in
6.9
ms
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
⚙️
ML Infra
Content type:
Blog
medium.com
·
5d
5 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
Exclusive: MotherDuck adds agentic
data
ingestion
to its cloud analytics service
🤖
AI
siliconangle.com
·
21h
21 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🗂️
RAG Systems
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Daily Deal: The 2026
Data
Engineering
Bundle featuring Databricks
📊
BigQuery
techdirt.com
·
16h
16 hours ago
Actions for Daily Deal: The 2026 Data Engineering Bundle featuring Databricks
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
💾
Cassandra
Content type:
Blog
medium.com
·
1d
1 day ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Data
Pipeline
Development: Building the Foundation of Modern
Data
Engineering
with SB Infowaves
⚙️
ML Infra
Content type:
Blog
medium.com
·
2h
2 hours ago
Actions for Data Pipeline Development: Building the Foundation of Modern Data Engineering with SB Infowaves
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
📊
BigQuery
Content type:
PDF
markokolarek.com
·
4d
4 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
📊
BigQuery
Content type:
Blog
cloud.google.com
·
16h
16 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
Linux Fundamentals for
Data
Engineering
📐
System Design
dev-to-uploads.s3.amazonaws.com
·
2d
2 days ago
·
DEV
Actions for Linux Fundamentals for Data Engineering
Franz
📨
Kafka
flathub.org
·
20h
20 hours ago
Actions for Franz
Introducing
Streamling
: Performant and Extensible
Data
Streaming
Framework
📨
Kafka
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
AI
Agents and the Fight for Customer
Data
🤖
AI
a16z.simplecast.com
·
6d
6 days ago
Actions for AI Agents and the Fight for Customer Data
Deploying Vector High-Performance Observability
Data
Pipeline
on Ubuntu 24.04
🔭
Observability
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
12h
12 hours ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
Calculating speed estimates with
Apache
Spark
🕸️
Distributed Systems
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Calculating speed estimates with Apache Spark
Senior
Data
Engineer
– Climate Friendly
☁️
Cloud Infra
au.seek.com
·
6d
6 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Multilayer Screening of Double and Conventional Perovskite Solar Cells Using SCAPS-1D and Machine Learning: Optimization of
ETL
, HTL, and Absorber for High-Efficiency Architectures
🤖
Machine Learning
Content type:
Academic
arxiv.org
·
6h
6 hours ago
Actions for Multilayer Screening of Double and Conventional Perovskite Solar Cells Using SCAPS-1D and Machine Learning: Optimization of ETL, HTL, and Absorber for High-Efficiency Architectures
Transit
Data
Ingestion
Platform Dev 101: Lyondle & Colosse
☁️
Cloud Infra
Content type:
Blog
johanmontorfano.com
·
1d
1 day ago
·
Lobsters
Actions for Transit Data Ingestion Platform Dev 101: Lyondle & Colosse
Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions
☁️
Cloud Infra
Content type:
Blog
aws.amazon.com
·
18h
18 hours ago
Actions for Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions
Embedding
pipelines
are the new
ETL
🗂️
RAG Systems
Content type:
Blog
infoworld.com
·
6d
6 days ago
Actions for Embedding pipelines are the new ETL
Introducing Flights: Agent-Native
Ingest
in MotherDuck
☁️
Cloud Infra
Content type:
Blog
motherduck.com
·
1d
1 day ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help
