⚙️ Data Engineering - tomasz · Scour

Dawn of the autonomous data pipeline 🎭Anthropic Claude

zyte.com·3d

Data Pipelines for Machine Learning: From Ingestion to Training (2026 Guide) 🤖Machine Learning

flexiana.com·1d

Solution for a high performance data lake 🏗️Build Systems

beacon.maris.nl·15h

From Spec-Driven Work to Work Orchestration 🔍Code Review

trilogyai.substack.com·7h·Substack

Article: Building Hierarchical Agentic RAG Systems: Multi-Modal Reasoning with Autonomous Error Recovery 🧠LLM Reasoning

infoq.com·1d

Learn data engineering by building a real project (sponsored tutorial + competition with prizes) 🔢NumPy

getbruin.com·3d·r/learnprogramming

mushroomsandchai/eia: Data pipeline orchestration for EIA datasets. 🏗️Build Systems

github.com·11h·DEV

Aspect oriented data quality for dataflows 📰Content Curation

docs.tabsdata.com·2d·Hacker News

Collaborative Analytics on Databricks 🥶Cold Start Problem

databricks.com·2d

Automate Your Data + ML Pipelines With Apache Airflow 🕷️Web Scraping

gitanjalisoni.medium.com·6d

mmid: Multi-Modal Integration and Downstream analyses for healthcare analytics in Python 📚Digital Humanities

arxiv.org·1d

Decision-first analytics 🚪User Onboarding

toolup.pro·3d

Show HN: I built a local data lake for AI powered data engineering and analytics 💾Local-first Software

stream-sock-3f5.notion.site·2d·Hacker News

Designing an Analytics Engine for AI Agent Workloads 🤖AI Agents

starrocks.io·4d

Versioned Analytics for Regulated Industries 🏗️Build Systems

datahike.io·5d

Software specialization is reversing – and it's raising the bar, not lowering it 🔍Code Review

dify.ai·6d·Hacker News

Microsoft ADO.NET Driver for Microsoft Fabric Data Engineering (Preview) | Microsoft Fabric Blog 📊Data Visualization

blog.fabric.microsoft.com·4d

Retail Analytics Market Strategic Insights, Technological Advancements, Growth Drivers, Opportunities and Leading Key Vendors To 2031 📈Data Science

gandhinagarnews.org·6d

meemeealm/Multithreaded-Ingestion-Pipeline: Mini ETL 🔧 ingestion pipeline which works in bronze layer 💻Creative Coding

github.com·2d·DEV

Semantic Data Processing with Holistic Data Understanding 🕸️Knowledge Graphs

arxiv.org·5d