Mistral releases the most detailed, independently audited environmental impact assessment of any LLM I've seen, their Large 2 (roughly equivalent to GPT-4o in capabilities). With any assessment, whether the impact is "big" or "bad" really depends on the frame you're using. They break out the impact between pretraining and inference, as is sensible, and give some comparison activities. For the average user, I think the news is quite heartening, a single query uses 0.05L of water (roughly 3 tablespoons) and emits the same amount of greenhouse gases as about 10 seconds of watching a livestream.
Obviously, inference is much costlier, but also is something that the typical user has almost no control over and is also a one-time fixed cost vs. the ongoing cost of inference. I expect these numbers to continue to fall.
Subscribe for new posts
Blog posts only. No commonplace entries. Never sold or shared.
Obviously, inference is much costlier, but also is something that the typical user has almost no control over and is also a one-time fixed cost vs. the ongoing cost of inference. I expect these numbers to continue to fall.