Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Engineering
📊 Data Engineering
data pipeline, data-intensive, stream processing, batch processing
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
233
posts in
11.2
ms
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
☕
Java
Content type:
Blog
cloud.google.com
·
6h
6 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
Introducing
Streamling
: Performant and Extensible
Data
Streaming
Framework
🌊
Stream Processing
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
☁️
Cloud Infrastructure
Content type:
PDF
markokolarek.com
·
4d
4 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
LakeQA
: An Exploratory QA Benchmark over a Million-Scale
Data
Lake
🌐
Distributed Systems
Content type:
Academic
arxiv.org
·
20h
20 hours ago
Actions for LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
Calculating speed estimates with
Apache
Spark
🌊
Stream Processing
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Calculating speed estimates with Apache Spark
Deploying Vector High-Performance Observability
Data
Pipeline
on Ubuntu 24.04
📐
API Design
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
2h
2 hours ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
Senior
Data
Engineer
– Climate Friendly
☁️
Cloud Infrastructure
au.seek.com
·
5d
5 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
🏛️
Software Architecture
Content type:
Blog
medium.com
·
15h
15 hours ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Announcing general availability of
Apache
Spark
4.0 on Amazon EMR
☁️
Cloud Infrastructure
Content type:
Blog
aws.amazon.com
·
1d
1 day ago
Actions for Announcing general availability of Apache Spark 4.0 on Amazon EMR
Exclusive: MotherDuck adds agentic
data
ingestion to its cloud analytics service
🦆
DuckDB
siliconangle.com
·
11h
11 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
Automating Real-time
Data
Pipelines
: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
☁️
Cloud Infrastructure
Content type:
Blog
medium.com
·
6d
6 days ago
Actions for Automating Real-time Data Pipelines: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🗄️
Databases
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Introducing Flights: Agent-Native Ingest in MotherDuck
🦆
DuckDB
Content type:
Blog
motherduck.com
·
1d
1 day ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
Embedding
pipelines
are the new
ETL
🦆
DuckDB
Content type:
Blog
infoworld.com
·
5d
5 days ago
Actions for Embedding pipelines are the new ETL
Claude Code for Research: Preventing Hallucinations
🦆
DuckDB
Content type:
News
Content type:
Blog
homeeconomics.substack.com
·
2d
2 days ago
·
Substack
Actions for Claude Code for Research: Preventing Hallucinations
15 years of Software Center – A Look in the Mirror and over the Front Windshield
🧑💻
Developer Experience
Content type:
Blog
metrics.blogg.gu.se
·
16h
16 hours ago
Actions for 15 years of Software Center – A Look in the Mirror and over the Front Windshield
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
📐
API Design
Content type:
Blog
medium.com
·
5d
5 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
AI
Security Best Practices for Regulated Industries
🔐
Security
orca.security
·
1d
1 day ago
Actions for AI Security Best Practices for Regulated Industries
Day 10 of 100 Days of ClickHouse®: What Makes ClickHouse SQL Different?
🏎️
ClickHouse
quantrail-data.com
·
19h
19 hours ago
·
DEV
Actions for Day 10 of 100 Days of ClickHouse®: What Makes ClickHouse SQL Different?
DuckDB Storage
Engine
for MariaDB. When the Sea Lion Learns to Quack.
🦆
DuckDB
mariadb.org
·
1d
1 day ago
·
Hacker News
Actions for DuckDB Storage Engine for MariaDB. When the Sea Lion Learns to Quack.
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help