Evaluate Your Agentic Tooling (opens in new tab)

Status: WIP tl;dr: Evaluate all your agentic tools in realistic end-to-end agentic tasks. Claims about token reduction from tools doesn’t transfer from experimental conditions to all agentic workflows. At the time of writing, agents also prefer their own tools and workflows, and users should not expect tools to have their intended effect without additional usage enforecement. Intro Organizations are entering the tokenmaxxing hangover stage now. And with that, lots of tooling is popping up cla...