The artificial intelligence (AI) boom’s ever-growing demand for compute and memory may mean price hikes even for those not interested in large language models (LLMs) and the like — with dynamic RAM (DRAM) component pricing on-track to triple in the next few months, an increase single-board computer (SBC) manufacturers will be unable to swallow.
The AI industry is growing at an ever-increasing rate, primarily due to generative AI technologies — in particular large language models (LLMs), massive machine learning models trained on vast quantities of data to convert inputs into a stream of tokens and return the most statistically-likely continuation tokens. When it works well, this appears to the user as a “thinking” and “reasoning” machine that can answer almost any question; sc…
The artificial intelligence (AI) boom’s ever-growing demand for compute and memory may mean price hikes even for those not interested in large language models (LLMs) and the like — with dynamic RAM (DRAM) component pricing on-track to triple in the next few months, an increase single-board computer (SBC) manufacturers will be unable to swallow.
The AI industry is growing at an ever-increasing rate, primarily due to generative AI technologies — in particular large language models (LLMs), massive machine learning models trained on vast quantities of data to convert inputs into a stream of tokens and return the most statistically-likely continuation tokens. When it works well, this appears to the user as a “thinking” and “reasoning” machine that can answer almost any question; scratch the surface, though, and it’s a parlor trick entirely disconnected from such lofty concepts as “truth” and “reality.”
The AI boom has demand for DRAM outstripping supply, with devices like the NVIDIA DGX Spark coming with 128GB of RAM as standard. (📷: NVIDIA)
The tendency for LLMs to confidently return convincing responses that are nonetheless absolute nonsense is a problem as-yet unsolved, with many companies simply opting to make their models ever-larger to compensate — an unsustainable approach, and one which has AI companies as thirsty for memory as they are for compute. NVIDIA’s last-generation edge-AI machine, the Jetson AGX Orin, came with a generous-for-the-time 32GB of RAM; its latest model, the DGX Spark, comes with 128GB. The servers used for training and the deployment of centralized AI-as-a-service platforms, meanwhile, require even more RAM: NVIDIA’s Rubin CPX, designed for models processing over a million tokens at a time, has 128GB of memory per node and is designed to be installed and operated in racks of 144 nodes, plus a further 144 non-CPX Rubin nodes and 36 Vera CPUs totaling 100TB (around 102,400GB) of RAM.
Memory production is not scaling to meet demand, and with multi-year construction required to build new semiconductor fabs is unlikely to grow quickly enough any time soon. The inevitable consequence: price hikes. Market watcher TrendForce has reported a 171.8% increase in contract pricing for DRAM components, a trend that seems likely to continue to deliver a tripling in production cost over the coming months. While that’s something the trillion-dollar-scale AI market can absorb, it will leave less well-heeled markets starved — including the single-board computer (SBC) market.
A near-tripling of DRAM contract pricing means component prices will rise too, foreshadowing price hikes for single-board computers like the Raspberry Pi 500+. (📷: Gareth Halfacree)
Recent years have seen SBCs, like mainstream computers, launching with ever-growing memory capacities: the recently-launched Raspberry Pi 500+ wedge-style computer, built around the same components as the Raspberry Pi 5, comes with 16GB of RAM as standard. Profit margins on SBCs are typically tight, and there’s little chance that when the new contract pricing bleeds through into off-the-shelf component pricing companies like Raspberry Pi and competitors will have enough leeway to absorb even a fraction of the increase — which means price hikes are inevitable.
For those in the market for a new SBC, then, one thing is clear: if your project needs anything more than the bare minimum of RAM, you’d be better buying now than later if you want the best value for money.
Freelance journalist, technical author, hacker, tinkerer, erstwhile sysadmin. For hire: freelance@halfacree.co.uk.