Gartner warns of substantial cost increases for agentic AI applications. This could amount to a fivefold increase by 2028.

 

Gartner analysts point to the typical multi-step execution of agentic solutions. Managing inference costs—the cost per interaction when a model is in production—is becoming a particular challenge. According to Will Sommer, Sr. Director Analyst, innovation is moving so fast that each subsequent generation of AI capability requires more, and often more expensive, tokens. The total cost of AI usage is rising without clarity regarding future, proportionate, and predictable value. Gartner calls this the Inference Paradox.

Sommer explains the Inference Paradox using the difference between a simple chatbot and an AI agent. A simple chatbot must read and interpret a question and respond quickly with a probabilistically reasonable answer. An AI agent must constantly reason, negotiate, and question itself. These tasks, collectively considered as agentic reasoning, increase costs by at least a factor of five, and often much more as task complexity increases. This means that companies seeking to achieve ROI with advanced AI applications must realize much higher returns or savings than with simple AI applications.

An aspect that Gartner does not mention, however, but which contributes to even less cost control, is the shift from fixed subscriptions to pay-per-use models at a number of AI providers. Other costs, such as those for security—essential for agentic AI—are also not included.

(Gartner)

Technology