Skip to main content
Scour
Browse
Getting Started
Login
Sign Up
You are offline. Trying to reconnect...
Copied to clipboard
Unable to share or copy to clipboard
Data Engineering
📊 Data Engineering
big data, data pipelines, ETL, analytics, BigQuery
Filter Results
Timeframe
Fresh
Past Hour
Today
This Week
This Month
Feeds to Scour
Subscribed
All
Scoured
214
posts in
6.5
ms
Deep dive: How Lightning
Engine
delivers 4.9x faster
Apache
Spark
performance
🗄️
Data Platforms
Content type:
Blog
cloud.google.com
·
23h
23 hours ago
Actions for Deep dive: How Lightning Engine delivers 4.9x faster Apache Spark performance
New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
🛠️
Infrastructure
Content type:
PDF
markokolarek.com
·
5d
5 days ago
·
Hacker News
Actions for New comment by mkolarek in "Ask HN: Who wants to be hired? (June 2026)"
LakeQA
: An Exploratory QA Benchmark over a Million-Scale
Data
Lake
📊
Benchmarking
Content type:
Academic
arxiv.org
·
1d
1 day ago
Actions for LakeQA: An Exploratory QA Benchmark over a Million-Scale Data Lake
GetCassis/dbt-agent-readiness
: Audit a
dbt
project for what an
AI
agent will get wrong if you point it at the
data
today.
🗄️
Data Platforms
Content type:
Code
github.com
·
6h
6 hours ago
·
Hacker News
Actions for GetCassis/dbt-agent-readiness: Audit a dbt project for what an AI agent will get wrong if you point it at the data today.
Exclusive: MotherDuck adds agentic
data
ingestion to its cloud
analytics
service
🗄️
Data Platforms
siliconangle.com
·
1d
1 day ago
Actions for Exclusive: MotherDuck adds agentic data ingestion to its cloud analytics service
DWH
Spark
MCP: Your Agent Can Read
Spark
History Now
🗄️
Data Platforms
Content type:
Blog
medium.com
·
5h
5 hours ago
Actions for DWH Spark MCP: Your Agent Can Read Spark History Now
SDLC vs. AIDLC: Why
Data
Engineering
is Pushing the Boundaries of Software Development
🗄️
Data Platforms
Content type:
Blog
medium.com
·
6d
6 days ago
Actions for SDLC vs. AIDLC: Why Data Engineering is Pushing the Boundaries of Software Development
From
BigQuery
to Live Maps: Building a Real-Time
AI
Fitness Agent
📊
Benchmarking
Content type:
Blog
medium.com
·
15h
15 hours ago
Actions for From BigQuery to Live Maps: Building a Real-Time AI Fitness Agent
Introducing Flights: Agent-Native Ingest in MotherDuck
🗄️
Data Platforms
Content type:
Blog
motherduck.com
·
1d
1 day ago
Actions for Introducing Flights: Agent-Native Ingest in MotherDuck
I Built a Zero-Cost Customer Churn Prediction Platform on GCP — Here’s Exactly How
🗄️
Data Platforms
Content type:
Blog
medium.com
·
4h
4 hours ago
Actions for I Built a Zero-Cost Customer Churn Prediction Platform on GCP — Here’s Exactly How
Deploying Vector High-Performance Observability
Data
Pipeline
on Ubuntu 24.04
🔭
Observability
Content type:
Reference
Content type:
Tutorial
docs.vultr.com
·
19h
19 hours ago
·
DEV
Actions for Deploying Vector High-Performance Observability Data Pipeline on Ubuntu 24.04
Run an
Apache
Airflow
DAG with Docker Compose and PostgreSQL
🖥️
HPC
pyimagesearch.com
·
3d
3 days ago
Actions for Run an Apache Airflow DAG with Docker Compose and PostgreSQL
Introducing Streamling: Performant and Extensible
Data
Streaming Framework
🏗️
Systems Design
Content type:
News
streamingdata.tech
·
2d
2 days ago
Actions for Introducing Streamling: Performant and Extensible Data Streaming Framework
Data
Pipeline
Development: Building the Foundation of Modern
Data
Engineering
with SB Infowaves
🗄️
Data Platforms
Content type:
Blog
medium.com
·
9h
9 hours ago
Actions for Data Pipeline Development: Building the Foundation of Modern Data Engineering with SB Infowaves
Announcing general availability of
Apache
Spark
4.0 on Amazon EMR
🗄️
Data Platforms
Content type:
Blog
aws.amazon.com
·
2d
2 days ago
Actions for Announcing general availability of Apache Spark 4.0 on Amazon EMR
Senior
Data
Engineer
– Climate Friendly
🛠️
Infrastructure
au.seek.com
·
6d
6 days ago
·
Hacker News
,
Hacker News
Actions for Senior Data Engineer – Climate Friendly
Upriver raises $14M to fix the unglamorous layer where enterprise
AI
quietly breaks: the
data
🗄️
Data Platforms
Content type:
News
thenextweb.com
·
6h
6 hours ago
Actions for Upriver raises $14M to fix the unglamorous layer where enterprise AI quietly breaks: the data
Calculating speed estimates with
Apache
Spark
📊
Benchmarking
Content type:
Blog
mapbox.com
·
3d
3 days ago
Actions for Calculating speed estimates with Apache Spark
IPO-bound
Databricks
reportedly eyes $175B valuation after hitting $5.4B revenue run rate — TFN
🗄️
Data Platforms
techfundingnews.com
·
2d
2 days ago
Actions for IPO-bound Databricks reportedly eyes $175B valuation after hitting $5.4B revenue run rate — TFN
Linux Fundamentals for
Data
Engineering
🐧
Linux Kernel
dev-to-uploads.s3.amazonaws.com
·
3d
3 days ago
·
DEV
Actions for Linux Fundamentals for Data Engineering
Page 2 »
Log in to enable infinite scrolling
Keyboard Shortcuts
Navigation
Next / previous item
j
/
k
Open post
o
or
Enter
Preview post
v
Post Actions
Love post
a
Like post
l
Dislike post
d
Undo reaction
u
Save / unsave
s
Recommendations
Add interest / feed
Enter
Not interested
x
Go to
Home
g
h
Interests
g
i
Feeds
g
f
Likes
g
l
History
g
y
Changelog
g
c
Settings
g
s
Browse
g
b
Search
/
Pagination
Next page
n
Previous page
p
General
Show this help
?
Submit feedback
!
Close modal / unfocus
Esc
Press
?
anytime to show this help
