Karpathy's Autoresearch found a 3-year-old bug in Posthog's query engine (and improved performance by 11%) (9 minute read) (opens in new tab)

Posthog pointed an AI agent at its query agent, fed it slow queries from production, and let it run overnight. The next morning, the team discovered that for almost three years, every query with a timestamp filter had not been using ClickHouse's primary key correctly. Fixing the issue cut the number of granules ClickHouse had to scan by 62% on the benchmark query and made the query itself meaningfully faster. This post discusses the team's setup, the bug itself, and PostHog's plans to automat...