Teams compare providers on token rates and then discover that tokens were the smallest line on the invoice. The rate card is real, and it is rarely where the money goes.
The proportions vary enormously by project, and at very high volume token spend can dominate. But for the common case of a feature inside a product, the salary cost of the people building and maintaining it usually exceeds the API bill by a wide margin, and a 20% difference in token price rarely changes the total picture.
Integration, prompt development, error handling, retry logic, structured output parsing and the iteration that follows first contact with real users. This is ordinary software work and it is costed like ordinary software work.
Provider choice affects it mainly through documentation quality, SDK maturity and how much of the surrounding machinery you have to build yourself.
Building a test set, scoring outputs and rerunning it when models change. Teams commonly skip this at the start and construct it after a bad release, at higher cost and under pressure.
It is also what makes a provider switch cheap, so it pays for itself the first time you consider one.
Logging prompts and responses, tracking cost per feature, watching latency and catching quality drift. Standard practice for any external dependency, and easy to forget for one that fails quietly.
Note that logging prompts and responses has data implications of its own, which need to match whatever policy governs the underlying material.
Where output is checked before it is used, that time is a running cost of the feature. For high-stakes output it does not go away as models improve, because the failure mode stays invisible.
An honest business case counts this. Features justified on the assumption that review disappears tend to disappoint.
Models are deprecated and replaced. Assume at least one migration a year per model you depend on, including prompt adjustment, re-evaluation and redeployment.
This is the cost most often absent from a first-year budget and most reliably present in the second.
Terms review, data protection assessment, security questionnaires and internal approval. Sometimes weeks of elapsed time, and largely independent of which provider is chosen.
Buying through an existing cloud agreement can reduce this substantially, which is why procurement route often decides provider.
If tokens are a minority of total cost, then a provider that is 20% cheaper per token but requires more engineering to work with is not cheaper. And a decision agonised over for weeks may be worth less than the weeks spent on it.
The factors worth weighting heavily are the ones that move the large lines: documentation and tooling quality, how easy the provider is to swap out later, how stable the model line has been, and whether you can buy it without a three-month procurement cycle. Token price matters most at genuine scale, where it becomes the dominant line and deserves the attention it usually gets prematurely.
None of this recommends a provider. It suggests that the comparison most teams run is weighted toward the number that is easiest to look up rather than the one that decides the budget.