The inherent weaknesses of large language models are reason enough to explore other technologies, such as reinforcement learning or recurrent neural networks. Credit: IDG-Owned We need to have a frank conversation about large language models (LLMs). At their core, LLMs are nothing more than sophisticated memorization machines, capable of reasonable-sounding statements, but unable to understand fundamental truth. Importantly and despite the fervent hopes of many, they are far from delivering or even prefiguring artificial general intelligence (AGI). The hype surrounding LLMs has reached stratospheric levels, fostering a misguided belief in their potential as AGI precursors. We find ourselves at a critical juncture where the erroneous linkage between LLMs and AGI threatens to slow down—not accelerate—genuine progress in artificial intelligence. The clamor for LLMs to evolve into AGI solutions epitomizes tunnel vision at its finest. Consider the vast investments poured into training ever-larger models, yielding only marginal improvements in tasks that are not text-based. Let’s face it: LLMs are not learning how to do mathematics. Their forte lies in tackling statistical text tasks with finesse. It’s imperative that we recalibrate expectations and acknowledge that although LLMs excel in certain domains, they fall short in others. To chart a course towards meaningful advancements in AI, we must sever the umbilical cord between LLMs and AGI. Contrary to popular belief, LLMs are not the gateway to AGI; if anything, they represent a detour (or a freeway off-ramp as Yann LeCun, chief AI scientist at Meta, recently said). Thinking beyond LLMs One of the hurdles in dispelling misconceptions about LLMs stems from their ubiquitous adoption among developers. Integrated seamlessly into developer tools, LLMs serve as invaluable autocomplete companions, effortlessly assisting developers in their coding endeavors. Even for coders, LLMs have both strengths and weaknesses. We should continue to take advantage of the former and avoid the latter. Last Friday the U.S. House banned staffers’ use of Microsoft’s AI-based Copilot software coding assistant because of concerns it could lead to data leaks. Microsoft told reporters it’s working on another version to better meet government security needs. Of course, developer-oriented AI isn’t simply a question of LLMs. Despite all the focus on LLMs, there are complementary AI approaches helping developers, too. But these solutions face headwinds in the market from LLMs. For example, critics of reinforcement learning technology claim it’s not true generative AI, citing its independence from LLMs. Yet, examples abound in the AI landscape, from DALL-E to Midjourney, where generative AI thrives without reliance on LLMs. Diffblue, as I’ve covered before, writes Java unit tests autonomously and 250 times faster than human developers without an LLM. (It uses reinforcement learning.) Midjourney, with its diffusion model, is yet another testament to the diversity of approaches within the AI realm. In fact, it’s very possible that the next leap forward in AI may not emerge from LLMs, which are inherently constrained by their architecture that encodes and predicts tokens that represent chunks of text or pixels, floundering when confronted with mathematical or symbolic logic tasks. Undoubtedly, LLMs will constitute a facet of future AGI endeavors, but they won’t monopolize it. History has repeatedly shown that breakthroughs in algorithms catalyze paradigm shifts in computing. As Thomas Kuhn once explained, scientific progress isn’t linear; it’s punctuated by disruptive innovations (or paradigm shifts, a phrase he coined). The structure of AI revolutions Reflecting on recent advancements underscores this point. Neural networks for image recognition showed steady improvement but were nowhere near accurate enough to be useful until recurrent neural network (RNN) architectures were developed, which dramatically improved image recognition accuracy to the point that those networks could outperform humans. The advent of transformer architectures ushered in a similar dramatic improvement in neural networks making text predictions, leading directly to the LLM. Now we’re already in the era of diminishing returns: GPT-4 is reportedly 100 times the size of GPT3.5, and while it is a notable improvement, it certainly isn’t 100 times better. Indeed, the meteoric rise of LLMs may even harm innovation in the AI market, argued Tim O’Reilly in a recent opinion piece in The Information. He cautioned that a handful of deep-pocketed LLM investors threatens to distort the market, fueling a race for monopoly that inhibits product-market fit, thus harming customers. The implications are clear: the inflated investments in LLMs risk yielding diminishing returns. Funds diverted towards more diverse AI technologies could yield more substantial dividends. As we navigate the labyrinthine landscape of artificial intelligence, let’s heed the lessons of history: Progress thrives on diversity, not monoculture. The future of AI isn’t etched in stone; it’s waiting to be shaped by the ingenuity of pioneers willing to explore beyond the confines of LLMs. Related content analysis Beyond the usual suspects: 5 fresh data science tools to try today The mid-month report includes quick tips for easier Python installation, a new VS Code-like IDE just for Python and R users, and five newer data science tools you won't want to miss. By Serdar Yegulalp Jul 12, 2024 2 mins Python Programming Languages Software Development analysis Generative AI won’t fix cloud migration You’ve probably heard how generative AI will solve all cloud migration problems. It’s not that simple. Generative AI could actually make it harder and more costly. By David Linthicum Jul 12, 2024 5 mins Generative AI Artificial Intelligence Cloud Computing news HR professionals trust AI recommendations HireVue survey finds 73% of HR professionals trust AI to make candidate recommendations, while 75% of workers are opposed to AI making hiring decisions. By Paul Krill Jul 11, 2024 3 mins Technology Industry Careers how-to Safety off: Programming in Rust with `unsafe` What does it mean to write unsafe code in Rust, and what can you do (and not do) with the 'unsafe' keyword? The facts may surprise you. By Serdar Yegulalp Jul 11, 2024 8 mins Rust Programming Languages Software Development Resources Videos