It’s not news to anyone that there are concerns about AI’s rising energy bill. But a new analysis shows the latest reasoning models are substantially more energy intensive than previous generations, raising the prospect that AI’s energy requirements and carbon footprint could grow faster than expected.
As AI tools become an ever more common fixture in our lives, concerns are growing about the amount of electricity required to run them. While worries first focused on the huge costs of training large models, today much of the sector’s energy demand is from responding to users’ queries.
And a new analysis from researchers at Hugging Face and Salesforce suggests that the latest generation of models, which “think” through problems step by step before providing an answer, use considerably …
It’s not news to anyone that there are concerns about AI’s rising energy bill. But a new analysis shows the latest reasoning models are substantially more energy intensive than previous generations, raising the prospect that AI’s energy requirements and carbon footprint could grow faster than expected.
As AI tools become an ever more common fixture in our lives, concerns are growing about the amount of electricity required to run them. While worries first focused on the huge costs of training large models, today much of the sector’s energy demand is from responding to users’ queries.
And a new analysis from researchers at Hugging Face and Salesforce suggests that the latest generation of models, which “think” through problems step by step before providing an answer, use considerably more power than older models. They found that some models used 700 times more energy when their “reasoning” modes were activated.
“We should be smarter about the way that we use AI,” Hugging Face research scientist and project co-lead Sasha Luccioni told Bloomberg. “Choosing the right model for the right task is important.”
The new study is part of the AI Energy Score project, which aims to provide a standardized way to measure AI energy efficiency. Each model is subjected to 10 tasks using custom datasets and the latest generation of GPUs. The researchers then measure the number of watt-hours the models use to answer 1,000 queries.
The group assigns each model a star rating out of five, much like the energy efficiency ratings found on consumer goods in many countries. But the benchmark can only be applied to open or partially open models, so leading closed models from major AI labs can’t be tested.
In this latest update to the project’s leaderboard, the researchers studied reasoning models for the first time. They found these models use, on average, 30 times more energy than models without reasoning capabilities or with their reasoning modes turned off, but the worst offenders used hundreds of times more.
The researchers say that this is largely due to the way AI reasoning works. These models are fundamentally text generators, and each chunk of text they output requires energy to produce. Rather than just providing an answer, reasoning models essentially “think aloud,” generating text that is supposed to correspond to some kind of inner monologue as they work through a problem.
Be Part of the Future
Sign up to receive top stories about groundbreaking technologies and visionary thinkers from SingularityHub.
100% Free. No Spam. Unsubscribe any time.
This can boost the number of words they generate by hundreds of times, leading to a commensurate increase in their energy use. But the researchers found it can be tricky to work out which models are the most prone to this problem.
Traditionally, the size of a model was the best predictor of how much energy it would use. But with reasoning models, how verbose their reasoning chains are is often a bigger predictor, and this typically comes down to subtle quirks of the model rather than its size. The researchers say this is a key reason why benchmarks like this are important.
It’s not the first time researchers have attempted to assess the efficiency of reasoning models. A June study in Frontiers in Communication found that reasoning models can generate up to 50 times more CO₂ than models designed to provide a more concise response. The challenge, however, is that while reasoning models are less efficient, they are also much more powerful.
"Currently, we see a clear accuracy-sustainability trade-off inherent in LLM technologies," Maximilian Dauner, a researcher at Hochschule München University of Applied Sciences in Germany who led the study, said in a press release. "None of the models that kept emissions below 500 grams of CO₂ equivalent [total greenhouse gases released] achieved higher than 80 percent accuracy on answering the 1,000 questions correctly."
So, while we may be getting a clearer picture of the energy impacts of the latest reasoning models, it may be hard to convince people not to use them.
Edd is a freelance science and technology writer based in Bangalore, India. His main areas of interest are engineering, computing, and biology, with a particular focus on the intersections between the three.