Abstract:The rising energy demands of machine learning (ML), e.g., implemented in popular variants like retrieval-augmented generation (RAG) systems, have raised significant concerns about their environmental sustainability. While previous research has proposed green tactics for ML-enabled systems, their empirical evaluation within RAG systems remains largely unexplored. This study presents a controlled experiment investigating five practical techniques aimed at reducing energy consumption in RAG systems. Using a production-like RAG system developed at our collaboration partner, the Software Improvement Group, we evaluated the impact of these techniques on energy consumptio…
Abstract:The rising energy demands of machine learning (ML), e.g., implemented in popular variants like retrieval-augmented generation (RAG) systems, have raised significant concerns about their environmental sustainability. While previous research has proposed green tactics for ML-enabled systems, their empirical evaluation within RAG systems remains largely unexplored. This study presents a controlled experiment investigating five practical techniques aimed at reducing energy consumption in RAG systems. Using a production-like RAG system developed at our collaboration partner, the Software Improvement Group, we evaluated the impact of these techniques on energy consumption, latency, and accuracy. Through a total of 9 configurations spanning over 200 hours of trials using the CRAG dataset, we reveal that techniques such as increasing similarity retrieval thresholds, reducing embedding sizes, applying vector indexing, and using a BM25S reranker can significantly reduce energy usage, up to 60% in some cases. However, several techniques also led to unacceptable accuracy decreases, e.g., by up to 30% for the indexing strategies. Notably, finding an optimal retrieval threshold and reducing embedding size substantially reduced energy consumption and latency with no loss in accuracy, making these two techniques truly energy-efficient. We present the first comprehensive, empirical study on energy-efficient design techniques for RAG systems, providing guidance for developers and researchers aiming to build sustainable RAG applications.
| Comments: | Accepted for publication at the 2026 International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS’26) |
| Subjects: | Software Engineering (cs.SE) |
| Cite as: | arXiv:2601.02522 [cs.SE] |
| (or arXiv:2601.02522v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2601.02522 arXiv-issued DOI via DataCite (pending registration) | |
| Related DOI: | https://doi.org/10.1145/3786581.3786932 DOI(s) linking to related resources |
Submission history
From: Justus Bogner [view email] [v1] Mon, 5 Jan 2026 19:50:49 UTC (354 KB)