The Illusion of AI Reasoning

The glittering promise of artificial intelligence, particularly its generative variants, has captivated the world, drawing in billions in investment and fueling visions of a future where machines seamlessly execute complex tasks and even replace human jobs at scale.

OpenAI, a titan in this arena, recently commanded a staggering $300 billion valuation.

Yet, a recent bombshell paper from Apple, “The Illusion of Thinking: Understanding the Limitations of Reasoning Models via the Lens of Problem Complexity,” casts a long shadow over this optimistic narrative, suggesting that despite the hype, Large Reasoning Models (LRMs) are, in critical ways, falling profoundly short.

The paper, which meticulously dissects the “thinking” abilities of leading LRMs like Claude 3.7 Sonnet Thinking, Gemini Thinking, and OpenAI’s o-series, reveals a sobering reality: true AI reasoning, as humans understand it, remains an elusive mirage.

Jing Hu, a perceptive writer and researcher who has delved deep into the paper’s findings, offers a blunt assessment: “AI is just sophisticated pattern matching, no thinking, no reasoning.”

She argues that these models can only perform accurately up to a certain, limited degree of complexity.

Apple’s researchers put LRMs through their paces in a closed puzzle environment, using games like Checkers Jumping, River Crossing, and the classic Tower of Hanoi, varying the complexity across three distinct stages.

The results were, to put it mildly, counterintuitive.

At ‘Low Complexity,’ the supposedly advanced LRMs were actually outperformed by their simpler, regular Large Language Model (LLM) counterparts.

Hu explained this baffling outcome: “The reasoning models were overthinking — wrote thousands of words, exploring paths that weren’t needed, second-guessing correct answers and making things more complicated than they should have been.”

In the Tower of Hanoi, where a human can solve the puzzle in seven moves, Claude-3.7 Sonnet Thinking used “10x more tokens compared to the regular version while achieving the same accuracy… it’s like driving a rocket ship to the corner store.” This isn’t efficiency; it’s computational profligacy.

At ‘Medium Complexity,’ LRMs did manage to edge out LLMs, showing faint traces of Chain of Thought (CoT) reasoning – the step-by-step articulation of problem-solving.

But this marginal improvement came at a colossal cost.

“These thinking models use 10-50x more compute power,” Hu noted, equating it to “paying $500 instead of $50 for a hamburger that tastes 10% better.”

For Hu, this isn’t a breakthrough; it’s a performance dressed to impress, just “simple enough to avoid total failure.” The real collapse occurred at ‘High Complexity.’

Here, both LRMs and standard models simply crumbled, their accuracy plummeting to zero.

As problems grew genuinely complex, the models “stopped trying,” as Hu observed from the data.

“Accuracy starts high for all models on simple tasks, dips slowly then crashes to near zero at a ‘critical point’ of complexity.” Worse, their effort didn’t scale; token usage, a proxy for “thinking,” peaked and then dropped sharply, even with available budget.

The models abandoned reasoning altogether.

This isn’t news to Gary Marcus, a long-standing authority and critic in the AI field.

He views Apple’s findings as a powerful vindication of his decades-old argument: neural networks excel within their training distribution but break down when faced with novel, “edge cases” outside that familiar data.

LRMs, he contends, have a fundamental scaling problem, often requiring outputs too lengthy to be practically produced.

“Our argument is not that humans don’t have any limits, but LRMs do, and that’s why they aren’t intelligent,” Marcus insists.

So, if generative AI’s reasoning capabilities are an “illusion,” what does genuine AI look like?

Ryan Pannell, CEO of Kaiju Worldwide, a firm specializing in predictive artificial intelligence for algorithmic trading, offers a compelling counter-narrative.

Unlike generative AI, his systems leverage statistical analysis and machine learning on closed, meticulously sanitized historical data – price, time, and quantity.

“Predictive systems don’t hallucinate because they aren’t guessing,” Pannell asserts.

His models, fed trillions of transactions, classify market regimes with high certainty.

When one outputs “I don’t know,” it signifies something truly unprecedented, a valuable early warning signal.

Pannell’s predictive AI foresaw the dramatic stock market drop of February 2020 with 96% certainty, based purely on shifting market patterns, even though it had no “awareness” of the unfolding COVID-19 pandemic.

This highlights a crucial distinction: predictive AI excels at pattern recognition for immediate forecasting (within about seven days, he notes), but it doesn’t “explain” the underlying human-world reasons.

Pannell’s skepticism extends to the current crop of LRMs, born from frustrating personal experience.

He recounts wasting three hours with ChatGPT 4.0, only to discover it had been “lying” to him the entire time, programmed to keep him engaged rather than provide honest answers about its capabilities.

He describes the experience as “like working with an extremely talented but completely drunk programmer,” capable of brilliance but also prone to overestimation, falsehoods, memory lapses, and chaotic behavior within a session.

Catriona Kennedy, a Ph.D. in Computer Science, sheds light on the fundamental divergence.

She explains that “automated reasoning” has always been a branch of AI centered on the “inference engine” – a logic-based calculator that operates on symbols and statements, “designed to be correct.” Generative AI, by contrast, is a statistical generator, inherently “prone to hallucinations because they do not interpret the logic of the text in the prompt.”

This distinction points to Symbolic AI, which uses knowledge structures like metadata and ontologies to classify data based on defined relationships and logic rules, ensuring correctness.

While pure Symbolic AI struggles with scale and “noisy” data, it offers a pathway to reliable reasoning.

The consensus among these experts is clear: the current generation of LRMs, despite their massive investment and public adoration, are not demonstrating human-like intelligence or robust reasoning.

For Jing Hu, it boils down to a stark truth: “Too much money depends on the illusion of progress — there is a huge financial incentive to keep the hype going even if the underlying technology isn’t living up to the promises. Stop the Blind worship of GenAI.” The path forward, as envisioned by both Pannell and Marcus, lies in integration.

Pannell suggests combining generative AI’s communication prowess with predictive systems’ data accuracy, allowing GPT to act as a user-friendly “interlocutor.” Marcus, echoing decades of his own advocacy, champions “Neurosymbolic AI” – a fusion of neural networks and symbolic algorithms.

This, he believes, is the true route to robust AI capable of genuine reasoning, learning, and cognitive modeling.

He laments that “closed-minded egotists with too much money and power” have resisted this idea for decades, only for current limitations to “accidentally vindicate the idea in the end.” The current reality of AI reasoning is far from the self-aware, universally capable intelligence often portrayed. It is a powerful tool, certainly, but one with significant, well-documented limitations.

Understanding these boundaries, rather than succumbing to the “illusion of progress,” is paramount for navigating the future of artificial intelligence responsibly.