Title:Reevaluating Self-Consistency Scaling in Multi-Agent Systems
Abstract:This study examines the trade-offs of increasing sampled reasoning paths in self-consistency for modern large language models (LLMs). Earlier research with older models showed that combining multiple reasoning chains improves results before reaching a plateau. Using Gemini 2.5 models on HotpotQA and Math-500, we revisit those claims under current model conditions. Each configuration pooled outputs from varying sampled reasoning paths and compared them to a single chain-of-thought (CoT) baseline. Larger models exhibited a more stable and consistent improvement curve. The results confirm that per…
Title:Reevaluating Self-Consistency Scaling in Multi-Agent Systems
Abstract:This study examines the trade-offs of increasing sampled reasoning paths in self-consistency for modern large language models (LLMs). Earlier research with older models showed that combining multiple reasoning chains improves results before reaching a plateau. Using Gemini 2.5 models on HotpotQA and Math-500, we revisit those claims under current model conditions. Each configuration pooled outputs from varying sampled reasoning paths and compared them to a single chain-of-thought (CoT) baseline. Larger models exhibited a more stable and consistent improvement curve. The results confirm that performance gains taper off after moderate sampling, aligning with past findings. This plateau suggests diminishing returns driven by overlap among reasoning paths. Self-consistency remains useful, but high-sample configurations offer little benefit relative to their computational cost.
| Comments: | 7 pages, 3 figures |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2511.00751 [cs.AI] |
| (or arXiv:2511.00751v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2511.00751 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chiyan Loo [view email] [v1] Sun, 2 Nov 2025 00:36:49 UTC (176 KB)