Most of what you call "AI" costs almost nothing

Talk to most teams about their AI bill and they tell the same story. It went from $50 a month to $5,000 a month in a quarter. The model is the cost driver. The CFO wants a plan.

Look at the actual pipeline first.

Almost none of it needs an LLM. Extracting from an API, no tokens. Transforming with code, no tokens. Loading into a database, no tokens. Indexing for retrieval uses embeddings, and embeddings run at about two cents per million tokens, which is essentially free. Storage is cheap. The whole data movement layer, where most pipelines actually live, has near-zero AI cost.

The expensive part is the small slice that does synthesis: generating answers, drafting responses, analyzing unstructured content, summarizing. That’s where the LLM call lives, and that’s where most teams have accidentally routed everything, because they treated “AI” as one capability instead of a tiered set of operations.

A sane architecture classifies every operation by what kind of intelligence it actually needs. Pattern matching goes to regex or a cheap classifier. Entity extraction goes to a specialized model. Translation between known formats goes to a Haiku-class model at roughly a dollar per million input tokens. Only the synthesis of unstructured information into something useful gets the big model, with caching, batching, and confidence thresholds gating when it actually gets called.

A real multi-tenant platform handling serious data volume, done this way, sits at roughly $300 to $600 per tenant per month in AI costs. That’s what I see in my own builds. The teams burning through $20k a month are doing the same volume of AI on more expensive models, running synthesis-grade calls on operations that a classifier or a Haiku-class model would have handled for pennies. Their bill is a routing problem dressed up as a model-cost problem.

Routing is the architecture.