(Refer to the presentation title from co-presenter Abdul Karim) — Abdul Karim & Jack Silman at AI Engineer Melbourne 2026 (opens in new tab)

We Fired Our LLM Judge The idea seemed perfect: use an LLM to evaluate the outputs of other LLMs. Judge them on accuracy, helpfulness, tone, safety. LLM-as-judge became a popular approach, especially when human evaluation is expensive or impossible to scale. You could evaluate thousands of outputs quickly. You could do it continuously as your […]