Is AI scaling hitting a wall or still accelerating?
For five years, AI progress seemed like a straight line up: bigger models, more data, more compute, better results. Then in late 2024, reports emerged that the next generation of models wasn't delivering the expected leap. If the scaling laws are breaking, the entire AI industry narrative — and the trillion-dollar valuations built on it — is in trouble. If they're not, we're about to see capabilities that make GPT-4 look like a calculator.
Where They Stand
Scaling is hitting diminishing returns
Gary Marcus has been arguing since 2022 that the scaling paradigm would hit a wall, and he points to late-2024 reports as vindication. He highlights that GPT-4 to GPT-4o showed marginal improvements despite massive compute increases, that Google's Gemini Ultra underperformed expectations relative to training cost, and that the "throw more data and compute at it" approach is running into fundamental limitations: the internet has a finite amount of high-quality text, and each doubling of compute yields progressively smaller gains. François Chollet has provided the most rigorous technical critique through his ARC-AGI benchmark, which tests genuine novel reasoning. Despite years of scaling, LLM performance on ARC barely improved — until the o3 model applied massive test-time compute, which Chollet argues is a different paradigm from training-time scaling and has its own cost ceiling. He maintains that scaling alone cannot produce general intelligence because LLMs fundamentally interpolate within their training distribution rather than extrapolating beyond it. Yann LeCun has argued that the entire autoregressive LLM architecture is a dead end for achieving human-level intelligence, regardless of scale. He predicts the field will shift to fundamentally different architectures — what he calls "world models" and "objective-driven AI" — which amounts to admitting that the current scaling trajectory has a ceiling.
Scaling continues to work
Sam Altman has dismissed "scaling wall" narratives as premature, arguing that OpenAI's internal results show continued capability gains that haven't been publicly released. He has said repeatedly that the scaling laws hold, that compute investment continues to pay off, and that the o1 and o3 model series demonstrate a new scaling dimension — test-time compute — that opens another axis of improvement beyond pre-training scale alone. Dario Amodei has been more measured but equally bullish, noting that Anthropic's internal research shows no fundamental ceiling to scaling, only engineering challenges that more money and better infrastructure can solve. He has pointed to Claude's steady improvement across versions as evidence that each generation delivers meaningful capability gains. Jensen Huang, whose NVIDIA sells the GPUs that power scaling, has the most obvious financial incentive to promote continued scaling — but he backs it with data, showing that AI training compute is growing 4x per year and that each generation of NVIDIA hardware (H100 to B100 to Rubin) delivers 2-3x efficiency gains that extend the scaling runway. His argument: even if the "bits per FLOP" ratio is flattening, the total available FLOPs are growing so fast that raw capability continues to surge.
The paradigm is shifting, not stalling
Demis Hassabis has framed the moment not as a wall but as a transition. Google DeepMind's approach has always combined scaling with architectural innovation — AlphaFold didn't just scale a language model, it built a fundamentally new architecture for protein structure prediction. Hassabis argues that the next leaps in AI will come from combining LLM-style scaling with new paradigms: reinforcement learning from real-world interaction, multi-modal grounding, and what he calls "neuroscience-inspired architectures." Ilya Sutskever, after leaving OpenAI to found Safe Superintelligence Inc. in mid-2024, signalled through his new company's name and mission that he believes significant progress remains possible — but perhaps through fundamentally different approaches than pure pre-training scale. His departure from OpenAI and focus on safety-first capability research suggests he sees the path forward as narrower and more technically challenging than simply "bigger GPT." Noam Shazeer, the co-inventor of the transformer architecture who founded Character.AI and then returned to Google, has demonstrated through his work that architectural efficiency innovations (mixture of experts, speculative decoding, better attention mechanisms) can deliver capability gains that look like scaling even when raw compute doesn't increase. The paradigm-shift camp agrees that "just make it bigger" is running out of gas, but disagrees that this means progress is slowing — it means the field is evolving.
We haven't seen anything yet
Elon Musk, through xAI and Grok, has taken the position that AI capability growth is about to accelerate, not slow down. He has pointed to xAI's Colossus cluster — reportedly the largest AI training facility in the world — as evidence that the compute frontier is expanding faster than sceptics realise. His argument: the "plateauing" narrative is based on comparing publicly released models, while the most capable systems are being kept internal. He has predicted that AI will exceed any individual human's intelligence by late 2025 and all of humanity's combined intelligence by 2029. David Luan, CEO of Adept AI, brings a more technical perspective: most of the AI capability people interact with (ChatGPT, Claude) represents 2023-era technology. The models currently in training represent a fundamentally different capability level, and the public is evaluating the trajectory based on outdated data points. He argues that "agentic AI" — systems that can plan, use tools, and complete multi-step tasks autonomously — represents a capability unlock that makes previous benchmarks irrelevant. Both argue that declaring a plateau based on incremental chatbot improvements misses the architectural and infrastructure revolutions happening behind closed doors.
Patrick's Take
This is the debate where I'm most careful with my language, because it directly affects how Malaysian businesses plan their AI investment. If scaling is plateauing, then the AI tools available today are roughly representative of what you'll have for the next few years — which means you should invest heavily in mastering CURRENT tools rather than waiting for the next breakthrough. If scaling continues, then whatever you build today will be obsolete in 18 months — which means you should invest in adaptability and AI literacy rather than specific tool proficiency. Here's my honest read: both things are true simultaneously. The era of "just make the model bigger and it gets proportionally smarter" is probably ending. But the era of "make AI dramatically more capable through new techniques" is just beginning. Test-time compute, agentic workflows, specialised fine-tuning, multi-modal systems — these are delivering real capability gains that my clients notice in their day-to-day work. My practical advice to Malaysian SMEs: don't bet on a specific model or company. Bet on the CATEGORY. AI tools will continue to get meaningfully better every year for the foreseeable future. Build your workflows to be model-agnostic, train your team on principles rather than specific prompts, and budget for continuous learning. The plateau debate is for researchers. Your business should be focused on squeezing maximum value from what's already available — and most businesses I train are using about 10% of current capabilities.
What This Means for Your Business
The plateau debate translates into a concrete budget question: how much should you invest in AI infrastructure vs. AI education? If progress is slowing, invest more in deeply integrating current tools into your workflows — the ROI on mastering today's AI is higher when tomorrow's AI isn't dramatically different. If progress continues, invest more in flexible architectures and team adaptability. The practical sweet spot for most Malaysian SMEs: spend 70% of your AI budget on training and integration with current tools, 30% on staying current with new developments. Review quarterly. What you should NOT do is freeze — waiting for "the right time" to adopt AI because "better models are coming" is the single most expensive mistake I see businesses make. The best time to start was a year ago. The second best time is this week.
What to Actually Worry About
Whether AI is plateauing or accelerating, the worry for Malaysian businesses is the same: the adoption gap is widening. Every month you don't integrate AI into your operations, your AI-adopting competitors pull further ahead. If progress plateaus, early adopters who've mastered current tools have a durable advantage. If progress accelerates, early adopters who've built AI-native workflows will absorb new capabilities faster because their teams already think in AI-augmented terms. The businesses I'm most concerned about are the ones using the plateau narrative as an excuse to delay — "we'll wait until AI is really good." It's already really good. The gap between knowing that and acting on it is where most Malaysian SMEs are losing ground.
Featured Minds in This Debate
Gary Marcus
Professor Emeritus of Psychology & Neural Science, New York University
Francois Chollet
Software Engineer & AI Researcher, Google
Yann LeCun
VP & Chief AI Scientist, Meta
Sam Altman
CEO, OpenAI
Dario Amodei
CEO & Co-Founder, Anthropic
Jensen Huang
Founder, President & CEO, NVIDIA
Demis Hassabis
CEO & Co-Founder, Google DeepMind
Ilya Sutskever
Co-Founder & Chief Scientist, Safe Superintelligence Inc. (SSI)
Noam Shazeer
VP of Engineering, Gemini, Google DeepMind (formerly Character.AI)
Elon Musk
CEO & Founder, xAI / Tesla / SpaceX
David Luan
CEO & Co-Founder, Adept AI
Last updated: 2026-04-13
←Back to AI Minds Directory