Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Pipelines
🔄 Data Pipelines
Airflow, orchestration, ingestion, batch processing, streaming
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
345
posts in
4.2
ms
It's official: Fivetran +
dbt
Labs merge to build the
data
foundation for trustworthy
AI
agents (Sponsor)
🔧
dbt
fivetran.com
·
6d
6 days ago
Actions for It's official: Fivetran + dbt Labs merge to build the data foundation for trustworthy AI agents (Sponsor)
Exclusive: MotherDuck adds agentic
data
ingestion
to its cloud analytics service
🛠️
Data Engineering
siliconangle.com
·
7h
7 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🛠️
Data Engineering
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Daily Deal: The 2026
Data
Engineering
Bundle featuring Databricks
🛠️
Data Engineering
techdirt.com
·
2h
2 hours ago
Actions for Daily Deal: The 2026 Data Engineering Bundle featuring Databricks
Introducing
Streamling
: Performant and Extensible
Data
Streaming
Framework
🛠️
Data Engineering
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
LakeQA
: An Exploratory QA Benchmark over a Million-Scale
Data
Lake
🛠️
Data Engineering
Content type:
Academic
arxiv.org
·
16h
16 hours ago
Actions for LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
Enhancements to Managed Service for
Apache
Spark
clusters
🛠️
Data Engineering
Content type:
Blog
cloud.google.com
·
6d
6 days ago
Actions for Enhancements to Managed Service for Apache Spark clusters
aayush4vedi/drift-spark
:
Spark-native
embedding lifecycle- produce,
CDC
refresh, model-migrate, audit.
🛠️
Data Engineering
Content type:
Code
github.com
·
8h
8 hours ago
·
Hacker News
Actions for aayush4vedi/drift-spark: Spark-native embedding lifecycle- produce, CDC refresh, model-migrate, audit.
Linux Fundamentals for
Data
Engineering
🛠️
Data Engineering
dev-to-uploads.s3.amazonaws.com
·
2d
2 days ago
·
DEV
Actions for Linux Fundamentals for Data Engineering
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
🛠️
Data Engineering
Content type:
Blog
medium.com
·
5d
5 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
🛠️
Data Engineering
Content type:
Blog
medium.com
·
11h
11 hours ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Calculating speed estimates with
Apache
Spark
⚡
Apache Spark
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Calculating speed estimates with Apache Spark
Choosing the right workflow
orchestration
service for your use case: Amazon MWAA and AWS Step Functions
🛠️
Data Engineering
Content type:
Blog
aws.amazon.com
·
4h
4 hours ago
Actions for Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions
Introducing Flights: Agent-Native
Ingest
in MotherDuck
🛠️
Data Engineering
Content type:
Blog
motherduck.com
·
20h
20 hours ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
Transit
Data
Ingestion
Platform Dev 101: Lyondle & Colosse
🛠️
Data Engineering
Content type:
Blog
johanmontorfano.com
·
1d
1 day ago
·
Lobsters
Actions for Transit Data Ingestion Platform Dev 101: Lyondle & Colosse
Senior
Data
Engineer
– Climate Friendly
🛠️
Data Engineering
au.seek.com
·
5d
5 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Real-time
data
replication to your
data
warehouse
, self-serve
🛠️
Data Engineering
artie.com
·
1d
1 day ago
·
Hacker News
,
Hacker News
Actions for Real-time data replication to your data warehouse, self-serve
Integration Patterns: How To Choose for Your Architecture
🛠️
Data Engineering
Content type:
Blog
blog.n8n.io
·
2d
2 days ago
Actions for Integration Patterns: How To Choose for Your Architecture
Automating Real-time
Data
Pipelines
: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
🛠️
Data Engineering
Content type:
Blog
medium.com
·
6d
6 days ago
Actions for Automating Real-time Data Pipelines: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
Presentation: Beyond Prompting: Context
Engineering
and Memory Management for
AI
Systems at Scale
🛠️
Data Engineering
Content type:
News
infoq.com
·
8h
8 hours ago
Actions for Presentation: Beyond Prompting: Context Engineering and Memory Management for AI Systems at Scale
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help
