🌟 spark

optimisation, memory management, query plans, predicate push down, Livy

Feeds to Scour
SubscribedAll
Scoured 11644 posts in 595.8 ms
Apache Spark: The Engine Powering the Modern Data Revolution
netcomlearning.com·3d·
Discuss: DEV
🏗data engineering
Preview
Report Post
Building an Interpreter in Rust: A Journey Through Lexical Analysis, Parsing, and Evaluation
dev.to·6d·
Discuss: DEV
🔧Functional Programming
Preview
Report Post
Optimizing Repartitions in DataFusion: How I Went From Database Noob to Core Contribution - Apache DataFusion Blog
datafusion.apache.org·6d
📂Data Partitioning
Preview
Report Post
Building an Agentic AI Pipeline for ESG Reporting
analyticsvidhya.com·6d
🔍AI Detection
Preview
Report Post
Unnesting Scalar Subqueries into Left Outer Joins in SQL
dev.to·4d·
Discuss: DEV
🔍Query Optimization
Preview
Report Post
timescale/pgvectorscale: Postgres extension for vector search (DiskANN), complements pgvector for performance and scale. Postgres OSS licensed.
github.com·3d·
Discuss: Hacker News
💾Databases
Preview
Report Post
Context Mesh Lite: Hybrid Vector Search + SQL Search + Graph Search Fused (for Super Accurate RAG)
vmat.fillout.com·4d·
Discuss: DEV
💾Databases
Preview
Report Post
Build a Knowledge-Based Q&A Bot using Bedrock + S3 + DynamoDB/OpenSearch via AWS CDK
dev.to·3d·
Discuss: DEV
⚡DataFusion
Preview
Report Post
Optimizing LLM inference on Amazon SageMaker AI with BentoML’s LLM- Optimizer
aws.amazon.com·3d
📊Columnar Engines
Preview
Report Post
Profunctors, Arrows, & Static Analysis
elvishjerricco.github.io·5d·
Discuss: Hacker News
🔧Functional Programming
Preview
Report Post
No, Small Models Are Not the "Budget Option" (English)
mostlylucid.net·2h
📊Columnar Engines
Preview
Report Post
BigQuery Salesforce Integration: Getting Your Data Where It Needs to Be
gspann.com·10h·
Discuss: DEV
⚡DataFusion
Preview
Report Post
Embedding-Based Tool Selection for AI Agents
zarar.dev·6d·
Discuss: Hacker News
📊Columnar Engines
Preview
Report Post
Day 23: Spark Shuffle Optimization
dev.to·4d·
Discuss: DEV
🏗data engineering
Preview
Report Post
Show HN: I built the fastest AI app builder that I can find
vibes.higashi.blog·3d·
Discuss: Hacker News
📊Columnar Engines
Preview
Report Post
I Vibe‑Coded a Booking API—Then Made It Production‑Grade (Part 1)
dev.to·4d·
Discuss: DEV
🧮Apache Calcite
Preview
Report Post
The Engineering guide to Context window efficiency
dev.to·4d·
Discuss: DEV
🔄Feed Aggregation
Preview
Report Post
BRIDGE: Budget-aware Reasoning via Intermediate Distillation with Guided Examples
arxiv.org·3d
🏗data engineering
Preview
Report Post
Best way to annotate large parquet LLM logs without full rewrites?
news.ycombinator.com·5d·
Discuss: Hacker News
📊Column Stores
Preview
Report Post
The Anti-Cloud AI Manifesto: Meet “Ratio,” the DSL That Runs Game-Grade Intelligence on a Laptop
hackernoon.com·3d
⚙️Query Compilers
Preview
Report Post