AI-powered footprinting tools prone to misleading results

0
1
AI-powered footprinting tools prone to misleading results


Multiple services now offer to take a list of the materials that go into a product — or even just a description of it — and estimate the emissions generated by its manufacture.

The automation of the normally tedious process of creating product carbon footprints (PCFs) is one of the more tantalizing sustainability applications of artificial intelligence. But users of these services should treat the results with caution, suggests a study from researchers at Watershed, a carbon accounting platform. Impressive-sounding final results can mask significant errors in the AI’s thinking, the study concluded.

Time-saver

Automating PCF production is something many sustainability professionals would welcome. To measure a footprint, analysts typically identify the key components of a product and how much of each is used, then use industry data to convert that information into an emissions estimate. The intensive nature of the process, which can involve hunting down data from numerous suppliers, helps explain why PwC found that 69 percent of companies have created PCFs for less than a quarter of their product lineups.

Several companies are competing to alleviate that drudgery. Makersite says it will “automate accurate LCAs across your entire product portfolio in seconds,” while Terrascope claims it can achieve 70 percent accuracy without needing to trouble suppliers for data. Watershed also offers a product footprinting tool as part of its suite of services.

The outputs can appear impressive. Krishna Rao and his Watershed colleagues asked models from Anthropic, DeepSeek, Google and OpenAI to estimate PCFs for 175 products, from the materials, chemicals, textiles and other sectors; the best-performing model came within two multiples of the expert answer 77 percent of the time

Flawed analysis

But the estimates associated with each step can be as important as the final number. A manufacturer of an electronic device, for example, might want to reduce emissions by searching for a new vendor for a specific piece of hardware — a search that could be a waste of time if the emissions data for that component is flawed. And when the Watershed team had the models work through the steps  — decomposing the products into components and estimating the associated emissions — the accuracy plummeted to as low as 37 percent. “What surprised me most was the size of the gap,” said Rao.

The team designed the test as a benchmarking process that can be applied to specialist AI-powered PCF tools — including those offered by their employer. For now, Rao’s message to users is not not to rely on final results alone: “They have to understand the intermediate steps,” he said.