Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Engineering
🛠️ Data Engineering
data pipelines, ETL, data lakes, Apache Spark
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
294
posts in
6.9
ms
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
🔁
MLOps
Content type:
Blog
medium.com
·
5d
5 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
Daily Deal: The 2026
Data
Engineering
Bundle featuring Databricks
⚡
Apache Spark
techdirt.com
·
8h
8 hours ago
Actions for Daily Deal: The 2026 Data Engineering Bundle featuring Databricks
Introducing Streamling: Performant and Extensible
Data
Streaming Framework
⚙️
Query Engines
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
LakeQA
: An Exploratory QA Benchmark over a Million-Scale
Data
Lake
💬
LLMs
Content type:
Academic
arxiv.org
·
21h
21 hours ago
Actions for LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🗄️
Databases
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Snowflake
Datastream
:
Kafka-native
streaming in Snowflake
📨
Kafka
snowflake.com
·
6d
6 days ago
·
Hacker News
Actions for Snowflake Datastream: Kafka-native streaming in Snowflake
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
⚙️
Backend Engineering
Content type:
Blog
medium.com
·
16h
16 hours ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Linux Fundamentals for
Data
Engineering
⚡
Apache Spark
dev-to-uploads.s3.amazonaws.com
·
2d
2 days ago
·
DEV
Actions for Linux Fundamentals for Data Engineering
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
⚡
Apache Spark
Content type:
Blog
cloud.google.com
·
7h
7 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
Beyond Dual Writes: Microservice Integration Strategies
⚙️
Backend Engineering
Content type:
Blog
medium.com
·
2d
2 days ago
Actions for Beyond Dual Writes: Microservice Integration Strategies
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
🗄️
Databases
Content type:
PDF
markokolarek.com
·
4d
4 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
Franz
📨
Kafka
flathub.org
·
12h
12 hours ago
Actions for Franz
Real-time
data
replication to your
data
warehouse
, self-serve
🧊
Apache Iceberg
artie.com
·
1d
1 day ago
·
Hacker News
,
Hacker News
Actions for Real-time data replication to your data warehouse, self-serve
Location: Edmonton, Canada Remote: Yes Willing to relocate: Yes, within Canada T...
🧊
Apache Iceberg
Content type:
Discussion
news.ycombinator.com
·
4h
4 hours ago
·
Hacker News
Actions for Location: Edmonton, Canada Remote: Yes Willing to relocate: Yes, within Canada T...
Exclusive: MotherDuck adds agentic
data
ingestion to its cloud analytics service
☁️
Cloud Computing
siliconangle.com
·
12h
12 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
Senior
Data
Engineer
– Climate Friendly
🗄️
Databases
au.seek.com
·
5d
5 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Kafka
Share Groups and Parallelizing Consumption -
Part
3: Client-local parallelism
📨
Kafka
Content type:
Blog
jack-vanlightly.com
·
1d
1 day ago
Actions for Kafka Share Groups and Parallelizing Consumption - Part 3: Client-local parallelism
Deploying Vector High-Performance Observability
Data
Pipeline
on Ubuntu 24.04
📨
Kafka
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
3h
3 hours ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
What Went Wrong with
Data
Lakes
? A 15-Year Reality Check from the Field
🧊
Apache Iceberg
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for What Went Wrong with Data Lakes? A 15-Year Reality Check from the Field
Embedding
pipelines
are the new
ETL
🧠
AI Engineering
Content type:
Blog
infoworld.com
·
5d
5 days ago
Actions for Embedding pipelines are the new ETL
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help
