Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Engineering
🔧 Data Engineering
data pipelines, ETL, data warehouse, batch processing
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
281
posts in
4.7
ms
It's official: Fivetran +
dbt
Labs merge to build the
data
foundation for trustworthy
AI
agents (Sponsor)
🗄️
Databases
fivetran.com
·
6d
6 days ago
Actions for It's official: Fivetran + dbt Labs merge to build the data foundation for trustworthy AI agents (Sponsor)
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🗄️
Databases
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Daily Deal: The 2026
Data
Engineering
Bundle featuring Databricks
🏛️
ClickHouse
techdirt.com
·
5h
5 hours ago
Actions for Daily Deal: The 2026 Data Engineering Bundle featuring Databricks
Exclusive: MotherDuck adds agentic
data
ingestion to its cloud analytics service
🗄️
Databases
siliconangle.com
·
10h
10 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
What Went Wrong with
Data
Lakes
? A 15-Year Reality Check from the Field
🗄️
Databases
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for What Went Wrong with Data Lakes? A 15-Year Reality Check from the Field
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
🗄️
Databases
Content type:
Blog
cloud.google.com
·
5h
5 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
🏗️
Infrastructure
Content type:
Blog
medium.com
·
5d
5 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
Linux Fundamentals for
Data
Engineering
🗄️
Databases
dev-to-uploads.s3.amazonaws.com
·
2d
2 days ago
·
DEV
Actions for Linux Fundamentals for Data Engineering
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
🏛️
ClickHouse
Content type:
Blog
medium.com
·
14h
14 hours ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Deploying Vector High-Performance
Observability
Data
Pipeline
on Ubuntu 24.04
🏗️
Infrastructure
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
1h
1 hour ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
Introducing Flights: Agent-Native Ingest in MotherDuck
🗄️
Databases
Content type:
Blog
motherduck.com
·
23h
23 hours ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
Senior
Data
Engineer
– Climate Friendly
🗄️
Databases
au.seek.com
·
5d
5 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Calculating speed estimates with
Apache
Spark
🗄️
Databases
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Calculating speed estimates with Apache Spark
Introducing Streamling: Performant and Extensible
Data
Streaming Framework
⚡
Kafka
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions
🏛️
ClickHouse
Content type:
Blog
aws.amazon.com
·
7h
7 hours ago
Actions for Choosing the right workflow orchestration service for your use case: Amazon MWAA and AWS Step Functions
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
🗄️
Databases
Content type:
PDF
markokolarek.com
·
4d
4 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
Optimize
Spark
and
Databricks
jobs with
Datadog
🏗️
Infrastructure
Content type:
Blog
datadoghq.com
·
1d
1 day ago
Actions for Optimize Spark and Databricks jobs with Datadog
Automating Real-time
Data
Pipelines
: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
🏗️
Infrastructure
Content type:
Blog
medium.com
·
6d
6 days ago
Actions for Automating Real-time Data Pipelines: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
CSU Student of Distinction: Anthony Arthur
🚀
Career Growth
Content type:
Academic
csuohio.edu
·
12h
12 hours ago
Actions for CSU Student of Distinction: Anthony Arthur
Azerbaijani Central Bank set to adopt
data
Lakehouse
system in 2026
🏛️
ClickHouse
trend.az
·
1d
1 day ago
Actions for Azerbaijani Central Bank set to adopt data Lakehouse system in 2026
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help
