Carbon Footprint of LLMs

A real concern of Large Language Models (LLMs) is the amount of energy and water required for training and deploying these models. For example, in their 2024 report², Google admitted that their carbon output increased over 13% year-over-year primarily due to the increased energy usage of their customer-facing AI efforts, including the training and inference of their flagship Gemini LLM. In the report they admit:

Similarly, Microsoft in its 2024 Sustainability report³, admitted that it's Greenhouse Gas (GHG) Scope 3 emissions increased over 30% from its 2020 base year (page 11³).

As model parameters continue to increase to the billions of parameters, the associated energy to train these models as well as the energy requirements for actual model inference use by customers has increased quickly. The training of the ChatGPT 3.5 model is estimated to have used 1,287 MWh of electricity and produced 552 tons of CO2. ChatGPT 4.0 models are estimated to require 7,200 MWh of energy¹, significantly more than OpenAI's previous 3.5 model.

In a 2024 report⁵, Wong estimates that each query to ChatGPT 3.5 generates 4.32 grams of CO2 with over 10 million queries running on 30,000 GPUs, with a total daily CO2 generation of 43,200 kg.

How can we reduce the LLMs' Carbon Footprint?

In a 2024 report ⁴ released by Microsoft, four suggestions are presented to help Generative AI developers and users of these LLMs to reduce the environmental impact of these models. These suggestions are:

• Model Selection: Pre-trained models use significantly less power than training new models
• Model Improvement: Prompt engineering, RAG, and Fine-tuning can all be used to improve functionality of existing models without needed to train the models
• Model Deployment: Using Model-as-a-Service (MaaS), the costs and energy requirements are less because the MaaS infrastructure is typically optimized by the vendor. Model-as-a-Platform (MaaP) requires more customization and may be inefficient to use. Model Parameters can also be used to optimize the model performance while minimizing the energy requirements to use.
• Model Evaluation: When using these models, users should evaluate the costs and performance in order to assess the applicability of their models as well as evaluate the output of the models for safety and risk concerns.

Resources

• Assessing the environmental impact of large language models
• LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Models