A precise look at LLM reasoning limits—without hype or denial

A growing body of research suggests that transformer-based language models exhibit systematic weaknesses on certain classes of reasoning tasks. This does not justify the claim that “LLMs cannot reason.” But it does show that their reasoning abilities are fragile, distribution-dependent, and unevenly reliable.

The mistake in much of the current debate is treating reasoning as a binary property. The evidence instead points to a gradient: transformers perform well in some reasoning regimes and break down sharply in others. Understanding whereand why this happens matters more than rhetorical victory.

This article reviews what the evidence actually demonstrates, where claims exceed proof, and what remains unresolved.

---

What the Evidence Shows

GSM-Symbolic (Apple, ICLR 2025): Brittleness Under Perturbation

Apple’s GSM-Symbolic study tested LLMs on arithmetic word problems whose structure was preserved while surface features were altered.

Key findings:

• Changing only numerical values caused large performance drops
• Adding irrelevant information degraded accuracy by up to 65%
• Superficial perturbations affected outputs more than structural changes

Apple summarized this as “no evidence of formal reasoning.”

What this demonstrates:

• Transformer reasoning in arithmetic tasks is highly sensitive to distributional shifts
• Learned solutions are not invariant to irrelevant variation
• Performance reflects shallow generalization rather than rule stability

What it does not demonstrate:

• That transformers cannot reason in general
• That pattern learning and reasoning are mutually exclusive
• That these results extend to all reasoning domains (legal, causal, analogical)

This is evidence of fragile generalization, not proof of reasoning absence.

---

Formal Limits on Composition (Columbia / Berkeley, 2024)

Recent theoretical work proves that transformers cannot solve certain compositional problems—regardless of scale.

What is proven:

• For specific task families (certain SAT variants, deep compositional functions), transformers face architectural impossibility bounds
• These limits arise from attention and depth constraints, not training data

What is not proven:

• That all multi-step reasoning falls into this category
• That transformers cannot implement working memory or variable binding in practice
• That empirical reasoning success contradicts these proofs

These are real but narrow limits. They matter—but they are not universal impossibility results.

---

Faith and Fate (Allen Institute, NeurIPS 2023)

Faith and Fate tested transformers on increasingly complex compositional tasks.

Findings:

• Performance collapses sharply as depth increases
• Success correlates with exposure to similar computational patterns
• Novel compositions cause failure even when components are familiar

The authors argue transformers rely on “linearized subgraph matching.”

The key takeaway is not that transformers “only pattern match,” but that:

the patterns they learn are local, shallow, and weakly constrained by invariants.

---

The Core Issue: Robustness, Not Intelligence

Across all studies, a common failure mode emerges:

Transformers reason competently within distribution and collapse under perturbation.

Humans also fail outside familiar regimes—but typically gradually. Transformers often degrade catastrophically.

This reframes the debate:

• The problem is not whether transformers reason
• The problem is whether their reasoning is robust, compositional, and invariant

That is an engineering question, not a philosophical one.

---

What Transformers Consistently Struggle With

Task Class	Evidence	Confidence
Novel multi-digit arithmetic	GSM-Symbolic	High
Deep compositional reasoning	Faith & Fate, theory	High
Long-chain consistency	Multiple studies	Moderate–High
Irrelevance robustness	GSM-Symbolic	Moderate–High

These failures are systematic, not anecdotal.

---

What Transformers Demonstrably Do Well

At the same time, transformers succeed in tasks that require nontrivial reasoning:

Capability	Why It’s Nontrivial
Code generation & debugging	Requires compositional semantics
Legal & logical reasoning	Involves abstraction & constraint handling
Analogical reasoning	Requires structural mapping
Proof assistance (with tools)	Formal reasoning under scaffolding
Chain-of-thought prompting	Improves multi-step coherence

These capabilities cannot be dismissed as trivial memorization.

---

Chain-of-Thought Is Not an Epicycle—but It Is a Signal

Chain-of-thought works because it:

• Externalizes intermediate state
• Reduces implicit depth
• Supplies a pseudo-working memory

This suggests a crucial insight:

Transformer reasoning improves when we manually provide the state persistence it does not reliably maintain internally.

That aligns with:

• compositional depth limits,
• tool use success,
• and hybrid system performance.

The limitation is less about reasoning ability than about state stability and manipulation.

---

Hybrid Architectures: Not a Retreat, but a Decomposition

Neurosymbolic systems (e.g., AlphaGeometry) succeed by combining:

• neural perception and pattern learning
• symbolic constraint enforcement and memory

Notably, these systems still use transformers—just not alone.

This reflects a broader truth:

Reasoning is not a single capability. It is a bundle of separable functions.

Expecting one architecture to robustly solve all of them is historically unrealistic.

---

What Practitioners Should Infer

High confidence uses:

• synthesis, summarization, ideation
• code assistance (with review)
• structured generation
• retrieval + verification

Low confidence uses:

• novel deep reasoning
• correctness-critical logic
• autonomous decision-making
• adversarial or distribution-shifted inputs

The lesson is not “don’t use LLMs,” but don’t trust them where invariants matter.

---

The Honest Conclusion

Transformers exhibit real, well-documented limitations on certain compositional reasoning tasks—some empirically observed, some formally proven.

But claims that “transformers can’t reason” overreach. They do reason—just fragilely, unevenly, and without guarantees.

The field is not at a settled endpoint. Whether progress comes from architectural change, hybrid systems, or better scaffolding remains open.

What is clear:

• the limits are real,
• the successes are real,
• and intellectual humility is warranted.

The future is unlikely to be “transformers forever” or “transformers were a dead end.” It will almost certainly be more plural, more modular, and less ideologically tidy.

And that’s usually how progress actually looks.

The conclusions in this article draw on a mix of recent empirical studies, formal theoretical results, and hybrid-system demonstrations published between 2023 and 2025. Together, they define the current boundary of what is known—and what remains unresolved—about transformer-based reasoning.

Sources & References

Empirical Evidence of Reasoning Fragility

Apple AI Research — GSM-Symbolic (ICLR 2025)

Testing the Robustness of Mathematical Reasoning in Language Models

https://machinelearning.apple.com/research/gsm-symbolic

Demonstrates that transformer performance on arithmetic word problems degrades sharply under superficial perturbations (number changes, irrelevant information), indicating weak invariance and brittle generalization.

---

Allen Institute for AI — Faith and Fate (NeurIPS 2023)

On the Faithfulness of Transformers in Compositional Reasoning

https://arxiv.org/abs/2305.18654

Shows systematic collapse on compositional tasks as depth increases, with success tightly correlated to exposure to similar patterns during training.

---

Formal / Theoretical Limits

Columbia University & UC Berkeley (2024)

On the Limitations of the Transformer Architecture

https://arxiv.org/html/2402.08164v2

Provides mathematical proofs that transformers cannot solve certain families of compositional problems beyond fixed depth, regardless of scale or data.

---

Quanta Magazine (Jan 2025)

Chatbot Software Begins to Face Fundamental Limitations

Accessible synthesis of recent theoretical results on transformer limitations, with expert commentary and contextualization.

---

Evidence of Conditional Reasoning Capability

Chain-of-Thought Prompting (Wei et al.)

Chain-of-Thought Prompting Elicits Reasoning in Large Language Models

https://arxiv.org/abs/2201.11903

Shows that explicitly externalizing intermediate steps substantially improves multi-step reasoning performance, suggesting latent but fragile reasoning capacity.

---

Code Generation Benchmarks (Codex / Copilot)

Representative benchmarks summarized in OpenAI and GitHub evaluations

https://github.com/features/copilot

Demonstrates practical compositional reasoning in programming tasks, albeit with known correctness and robustness limitations.

---

Hybrid and Alternative Architectures

DeepMind — AlphaGeometry (2024)

Solving Olympiad Geometry Without Human Demonstrations

https://deepmind.google/discover/blog/alphageometry-an-olympiad-level-ai-system-for-geometry/

Illustrates how combining neural models (including transformers) with symbolic constraint solving enables robust formal reasoning in structured domains.

---

Meta AI — V-JEPA (2024–2025)

Joint Embedding Predictive Architectures for World Modeling

https://ai.meta.com/blog/v-jepa-yann-lecun-ai-model-video-joint-embedding-predictive-architecture/

Proposes an alternative paradigm focused on learning predictive world models rather than token sequences, aimed at supporting causal reasoning.

---

Mamba (State Space Models)

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

https://arxiv.org/abs/2312.00752

Introduces a scalable alternative to attention-based models, enabling longer contexts and efficient sequence modeling, though not a direct solution to reasoning robustness.

---

Neurosymbolic AI Survey (2025)

A Survey of Neurosymbolic Artificial Intelligence

https://arxiv.org/html/2501.05435v1

Comprehensive overview of hybrid approaches combining neural learning with symbolic reasoning, including current capabilities and open challenges.

---

Meta-Level Analyses & Surveys

Large Language Model Limitations Survey (2025)

LLLMs: A Data-Driven Survey of Evolving Research

https://arxiv.org/abs/2505.19240

Aggregates empirical findings across domains, identifying reasoning and compositional generalization as persistent open problems.

---