InfoQ Piece Argues LLM Reliability Is a Platform Problem, Not an App One

Production LLM failures keep getting patched at the application layer, and that is exactly why they keep recurring. That is the argument in a new InfoQ article by Aditya Mulik, who describes an AI agent that produced hallucinated recommendations inside an inventory system and the decision to stop treating the problem as a bug in that one product. His conclusion: the model stack belongs to platform engineering, alongside the databases, queues and identity services teams already share.

The failure mode Mulik describes is familiar to anyone who has shipped an agent that touches structured data. A recommendation agent that invents a SKU or misstates stock levels does not just return a bad answer — it can trigger downstream actions, erode trust with the operators reading its output, and generate support tickets that are hard to trace back to a prompt change. Fixing that inside a single application means the next team hits the same wall with no shared memory of what went wrong.

Mulik's proposed remedy is a shared LLM platform offering a small set of common services. A prompt registry with versioning lets teams roll prompts forward and back the way they version code, so a regression can be identified and reverted instead of debated. Schema enforcement constrains model output to a validated shape before it reaches business logic, which is the difference between an agent that fails loudly and one that quietly corrupts a recommendation.

The third leg is cost attribution: tracking token spend by request, team or feature rather than as one aggregate bill at the end of the month. That matters because LLM costs scale with usage in ways traditional compute does not, and without per-request accounting no team can tell whether a prompt tweak doubled inference spend. Mulik frames these as infrastructure primitives — the kind of thing a platform group owns once and every product team consumes.

The piece lands as more enterprises move agents from demos into systems of record, where an invented inventory number is not a curiosity but an operational incident. It also echoes a broader shift in how AI engineering is being organized: evaluation harnesses, guardrails, gateways and observability are consolidating into platform teams rather than being rebuilt per product. Mulik's contribution is grounding that shift in a concrete failure rather than an architectural preference.

What remains unsettled is how much of this belongs in a central platform versus the application. Schema enforcement and prompt versioning are easy to standardize; model routing, retrieval strategy and evaluation criteria are often domain-specific enough that a shared layer can become a bottleneck. The next question for teams adopting this playbook is where they draw that line — and who owns it when a shared prompt registry ships a regression to every product at once.