Eval-Driven Agent Development: How I Stopped Tuning Prompts on Vibes (opens in new tab)

Series context: This is a follow-up to . Earlier posts covered the pipeline overview, the Validate phase, agent state management, and rate-limit resilience. This one is about the part that holds the whole thing together: how I know whether a change to my harness actually made it better. The Question I Couldn't Answer I changed a prompt. The next run looked better. Did the prompt help, or did I get lucky? For a long time my honest answer was "I think so?" I'd tweak a slash command, run my pipe...