Elevator Pitch

• The article systematically investigates the reasoning strengths and limitations of Large Reasoning Models (LRMs), revealing that their apparent intelligence collapses beyond certain problem complexities despite producing detailed reasoning traces.

Key Takeaways

• LRMs outperform standard large language models only on medium-complexity tasks; they underperform on simple problems and both fail on highly complex ones.
• As puzzle complexity increases, LRMs initially ramp up their reasoning effort, but unexpectedly reduce it at higher complexities, indicating a critical scaling limit.
• LRMs do not reliably use explicit algorithms and often reason inconsistently, casting doubt on their true reasoning abilities.

Most Memorable Aspects

• The finding that LRMs' reasoning effort declines after a certain complexity threshold, even when token budgets are sufficient.
• Clear identification of three performance regimes highlighting when LRMs help or hinder performance.
• The use of controllable puzzle environments to precisely analyze reasoning traces beyond final answers.

Direct Quotes

• "Frontier LRMs face a complete accuracy collapse beyond certain complexities."
• "They exhibit a counter-intuitive scaling limit: their reasoning effort increases with problem complexity up to a point, then declines despite having an adequate token budget."
• "LRMs have limitations in exact computation: they fail to use explicit algorithms and reason inconsistently across puzzles."