<p><strong>In this episode, we enter the world of Large Reasoning Models (LRMs).</strong> </p><p>We explore advanced AI systems such as OpenAI’s o1/o3, DeepSeek-R1, and Claude 3.7 Sonnet Thinking—models that generate detailed &quot;thinking processes&quot; (Chain-of-Thought, CoT) with built-in self-reflection before answering. </p><p>These systems promise a new era of problem-solving. Yet, their true capabilities, scaling behavior, and limitations remain only partially understood.</p><p>By conducting systematic investigations in controlled puzzle environments—including the Tower of Hanoi, Checker Jumping, River Crossing, and Blocks World—we uncover both the strengths and surprising weaknesses of LRMs. </p><p>These environments allow precise control over task complexity while avoiding data contamination issues that often plague established benchmarks in mathematics and coding.</p><p>A striking finding: <strong>LRMs face a complete accuracy collapse beyond certain complexity thresholds.</strong> Paradoxically, their reasoning effort (measured in &quot;thinking tokens&quot;) first increases with complexity, only to decline after a point—even when token budgets are sufficient.</p><p>We identify three distinct performance regimes:</p><ul><li><p><strong>Low-complexity tasks</strong> – where standard Large Language Models (LLMs) still outperform LRMs.</p></li><li><p><strong>Medium-complexity tasks</strong> – where LRMs’ additional &quot;thinking&quot; shows a clear advantage.</p></li><li><p><strong>High-complexity tasks</strong> – where both LLMs and LRMs collapse entirely.</p></li></ul><p>Another challenge is <strong>“overthinking.”</strong> On simpler problems, LRMs often find correct solutions early but continue to pursue false alternatives, wasting computational resources. Even more surprising is their weakness in <strong>exact computation</strong>: they fail to leverage explicit algorithms, even when provided, and show inconsistent reasoning across different puzzle types.</p><p>This episode invites you to rethink assumptions about AI’s capacity for <strong>generalizable reasoning</strong>. What does it truly mean for a machine to &quot;think&quot; under increasing complexity? And how should these insights shape the next generation of AI design and deployment?</p><p> </p><p>Sources: Shojaee, P., Mirzadeh, I., Alizadeh, K., Horton, M., Bengio, S., &amp; Farajtabar, M. (2025). <strong>The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity</strong>. (Unpublished manuscript). <a href="https://arxiv.org/abs/2506.06941" target="_blank" rel="noopener noreferer">https://arxiv.org/abs/2506.06941</a><br> </p><p><strong>Disclaimer: </strong><em>This podcast is generated by Roger Basler de Roca (contact) by the use of AI. The voices are artificially generated and the discussion is based on public research data. I do not claim any ownership of the presented material as it is for education purpose only.</em></p><p><a href="https://rogerbasler.ch/en/contact/">⁠https://rogerbasler.ch/en/contact/</a></p>