The AI industry has started confusing consumption with intelligence. Bigger context windows became a feature war. More tokens became a sign of sophistication. Quietly, token usage became a proxy for progress.
That should concern us.
We are normalizing AI systems that repeatedly ask for the same context and use compute to solve problems they should already remember how to solve. The result is an emerging anti-pattern teams now describe as “token maxxing”: treating higher token consumption as evidence of deeper intelligence or better productivity. It isn’t. In many cases, it signals the opposite.
A stateless system is not intelligent simply because it generates a lot of activity. If anything, excessive token consumption often indicates that the model’s underlying architecture is failing.
I’ve seen this pattern before. We once measured engineering productivity through lines of code written. Then we learned that more code meant more complexity and more ways for systems to break. Mature engineering organizations eventually stopped rewarding volume and started
rewarding elegance, efficiency, and reliability instead. I believe AI systems are heading toward the same reckoning.
Stateless systems are creating artificial work
Right now, many teams are building workflows where the model spends more time rebuilding context than solving the actual problem. Every prompt starts from zero, every session requires rehydrating history, and orchestration layers inject more context and tools just to recreate the
understanding the model already had five minutes ago.
Ask a coding assistant about a bug you were debugging yesterday, and it behaves like the conversation never happened. You paste the same repository structure into multiple prompts because the system forgot it. You repeatedly explain the same internal APIs and rewrite prompts, not because the task changed, but because the model lost the thread. Then we wonder why token counts explode.
A working paper from the Stanford Digital Economy Lab states that agentic AI tasks consume 1,000x more tokens than standard code chat, driven by input tokens – because the agent must re-read the entire conversation history before every action. This creates a dangerous illusion. Teams start believing that the growing complexity of the interaction itself is proof that meaningful reasoning is happening. Large prompts and orchestration graphs look sophisticated. Huge token consumption starts feeling like computational seriousness. But often, the system is simply compensating for missing memory. And the person on the other end, the developer, the customer, the end user, is the one absorbing that cost in slower responses, broken context, and interactions that start over every time.
A surprising amount of what is marketed today as “agentic intelligence” is context-reconstruction overhead. A workflow that needs multiple agents and repeated prompt injection just to answer a deterministic question is not scaling intelligence. It is scaling inefficiency.
Bigger context windows are not the same thing as memory
This problem becomes even more obvious in enterprise environments where AI systems operate across fragmented tools, codebases, tickets, documents, chats, and operational systems. Without durable memory, every interaction becomes expensive reassembly work.
The irony is that software engineering solved versions of this problem decades ago. Databases do not recompute everything from scratch for every query because rebuilding context continuously is inefficient, expensive, and unnecessary. Yet many AI systems effectively operate like goldfish with enormous vocabularies.
The current obsession with context windows risks making this worse. Expanding the amount of information a model can consume is useful, but bigger context windows are not the same thing as memory. Feeding more tokens into a stateless system does not magically create continuity. It simply increases the temporary information the model must process before forgetting it again.
In their Tokenomics paper, researchers from the Data-driven Analysis of Software (DAS) Lab at Concordia University found that input tokens average 53.9% of total consumption, a cost created by re-reading accumulated context, not generating new answers. Developers should be careful not to confuse temporary context accumulation with durable intelligence. At some point, developers will stop asking how many tokens a workflow consumes and start asking why it needed so many in the first place.
AI development is becoming a systems design problem
Instead of treating AI primarily as a prompting problem, we need to start treating it as a systems design problem. The important questions become very different. How do we reduce redundant inference cycles? How do we maintain persistent context across sessions and preserve codebase memory over time?
These are infrastructure and architecture questions. Not prompt engineering tricks. In my experience, the teams making real progress have already figured that out.
Effective AI systems will likely start to look less like endlessly chatting assistants and more like memory-aware computational systems. They will preserve relationships between decisions, code changes, incidents, workflows, and operational history. They will understand continuity
without requiring developers to restate everything repeatedly. Most importantly, they will shift the value equation away from interaction volume and toward outcome quality. Because developers are not paid to generate tokens. They are paid to solve problems.
The future belongs to systems that remember
The current AI cycle rewards activity more visibly than outcomes. I see organizations celebrating AI activity rather than engineering outcomes. Teams increasingly measure progress through interaction volume: more prompts, more orchestration layers, more agents, and more generated output. In some cases, developers are spending more time managing AI than doing the work that actually matters – the architectural decisions, the product thinking, the customer impact.
The best infrastructure systems are often the ones you barely notice because they remove friction instead of creating ceremony. A truly intelligent development system should not require developers to constantly reconstruct context, supervise orchestration chains, or manage prompt gymnastics just to maintain continuity. For me, the best systems are the ones you barely notice. They remember enough to stop asking the same questions.
