RouteJudge: An Open Platform for Reproducible and Preference-Aware LLM Routing (opens in new tab)

We present RouteJudge, an online pairwise preference evaluation framework for LLM routing systems, with a public platform available at Different from model-level response evaluation, RouteJudge focuses on router-level decision quality. For each user query, multiple routing strategies independently recommend candidate models under the same model pool and budget constraints. The selected model responses are then presented to users through anonymou...