Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Engineering
📊 Data Engineering
data pipeline, data-intensive, stream processing, batch processing
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
236
posts in
6.5
ms
Enhancements to Managed Service for
Apache
Spark
clusters
☁️
Cloud Infrastructure
Content type:
Blog
cloud.google.com
·
6d
6 days ago
Actions for Enhancements to Managed Service for Apache Spark clusters
Introducing
Streamling
: Performant and Extensible
Data
Streaming
Framework
🌊
Stream Processing
Content type:
News
streamingdata.tech
·
1d
1 day ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
LakeQA
: An Exploratory QA Benchmark over a Million-Scale
Data
Lake
🌐
Distributed Systems
Content type:
Academic
arxiv.org
·
19h
19 hours ago
Actions for LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
Deploying Vector High-Performance Observability
Data
Pipeline
on Ubuntu 24.04
📐
API Design
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
1h
1 hour ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
Calculating speed estimates with
Apache
Spark
🌊
Stream Processing
Content type:
Blog
mapbox.com
·
2d
2 days ago
Actions for Calculating speed estimates with Apache Spark
It's official: Fivetran +
dbt
Labs merge to build the
data
foundation for trustworthy
AI
agents (Sponsor)
🏗️
Platform Teams
fivetran.com
·
6d
6 days ago
Actions for It's official: Fivetran + dbt Labs merge to build the data foundation for trustworthy AI agents (Sponsor)
Designing an
ETL
Application: Why I Started with a Modular Monolith Before Microservices
🏛️
Software Architecture
Content type:
Blog
medium.com
·
14h
14 hours ago
Actions for Designing an ETL Application: Why I Started with a Modular Monolith Before Microservices
Introducing Flights: Agent-Native Ingest in MotherDuck
🦆
DuckDB
Content type:
Blog
motherduck.com
·
23h
23 hours ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
Announcing general availability of
Apache
Spark
4.0 on Amazon EMR
☁️
Cloud Infrastructure
Content type:
Blog
aws.amazon.com
·
1d
1 day ago
Actions for Announcing general availability of Apache Spark 4.0 on Amazon EMR
Exclusive: MotherDuck adds agentic
data
ingestion to its cloud analytics service
🦆
DuckDB
siliconangle.com
·
10h
10 hours ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
☁️
Cloud Infrastructure
Content type:
PDF
markokolarek.com
·
4d
4 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🗄️
Databases
pyimagesearch.com
·
2d
2 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Senior
Data
Engineer
– Climate Friendly
☁️
Cloud Infrastructure
au.seek.com
·
5d
5 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Claude Code for Research: Preventing Hallucinations
🦆
DuckDB
Content type:
News
Content type:
Blog
homeeconomics.substack.com
·
2d
2 days ago
·
Substack
Actions for Claude Code for Research: Preventing Hallucinations
Automating Real-time
Data
Pipelines
: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
☁️
Cloud Infrastructure
Content type:
Blog
medium.com
·
6d
6 days ago
Actions for Automating Real-time Data Pipelines: Deploying Pub/Sub to BigQuery with Dataflow Custom Template…
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
☕
Java
Content type:
Blog
cloud.google.com
·
5h
5 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
AI
Security Best Practices for Regulated Industries
🔐
Security
orca.security
·
1d
1 day ago
Actions for AI Security Best Practices for Regulated Industries
15 years of Software Center – A Look in the Mirror and over the Front Windshield
🧑💻
Developer Experience
Content type:
Blog
metrics.blogg.gu.se
·
15h
15 hours ago
Actions for 15 years of Software Center – A Look in the Mirror and over the Front Windshield
Gene dependency-informed inference of response to targeted cancer therapies
🏎️
ClickHouse
Content type:
Academic
nature.com
·
2d
2 days ago
Actions for Gene dependency-informed inference of response to targeted cancer therapies
Embedding
pipelines
are the new
ETL
🦆
DuckDB
Content type:
Blog
infoworld.com
·
5d
5 days ago
Actions for Embedding pipelines are the new ETL
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help