LLM Token Pricing Is a Facilities Question

A published price list looks like a commercial document. Read it next to the hardware specifications, and LLM token pricing turns out to be something closer to a readout of physics — the structure of the tariff follows the structure of the machine so precisely that you can infer one from the other.

Two features of every major provider's price sheet make the point. Output tokens cost several times what input tokens cost. And batch processing is discounted by half. Neither is a marketing decision. Both are consequences of how accelerators actually execute inference, and once you see that, the rest of the pricing landscape becomes considerably easier to reason about.

Underneath the list prices, which competition sets, is a floor that competition does not set. That floor is made of power contracts, grid connections and facility capacity — which is why efficiency gains reach customers quickly and cost increases arrive slowly.

The quick answer

Output tokens are priced at roughly five times input tokens because generating a token is memory-bound and sequential, while processing prompt tokens is compute-bound and parallel. Batch processing costs half because delayed execution lets the provider fill large batches, which is the single mechanism that makes accelerators efficient at decode. Beneath those list prices sits a cost floor set by energy, facility overhead and utilisation — and the energy component is under sustained upward pressure from demand growing far faster than grid capacity.

Key takeaways

• Across providers, output is priced at four to six times input — most often exactly five.
• That ratio tracks the prefill/decode asymmetry in the hardware, not a margin decision.
• Batch discounts of 50% are payment for the right to schedule, which raises arithmetic intensity.
• The IEA estimates data centres used around 415 TWh in 2024, about 1.5% of global electricity, heading for roughly 945 TWh by 2030.
• Accelerated servers are projected to grow at 30% a year against 9% for conventional servers.
• The IEA's high-growth case notes that "long grid connection queues" push additional supply toward fossil fuels — a constraint on capacity, not just on emissions.

What the price sheet reveals

Take the published rates as of September 2026. On Anthropic's price list, Haiku 4.5 is $1 per million input tokens and $5 per million output; Sonnet 5 is $2 and $10; Opus 5 is $5 and $25; Fable 5.1 is $10 and $50. On OpenAI's, gpt-5.6-luna is $0.20 and $1.20; gpt-5.6-terra $2 and $12; gpt-5.6-sol $4 and $20; gpt-6-astra $10 and $50.

Different companies, different architectures, different model families. The same ratio.

When two competitors independently converge on the same multiple across an entire product range, the multiple is describing a shared constraint rather than a shared strategy.

Why output costs more than input

Inference has two phases that behave nothing alike.

Processing the prompt — prefill — handles every input token in parallel. It is a small number of large matrix multiplications, high arithmetic intensity, and it uses the accelerator's tensor cores close to their rated capability. Input tokens are cheap because the hardware is good at this.

Generating the response — decode — produces one token at a time, each depending on the last. To emit a single token the accelerator must read the model's active weights from memory and perform roughly two operations per parameter. As set out in why memory bandwidth beats peak FLOPS, arithmetic on NVIDIA's published GB200 NVL72 figures gives a machine balance around 1,250 FLOPs per byte, while single-sequence decode supplies single digits. The compute sits idle waiting on memory.

Input tokens buy you the part of the workload the hardware is good at. Output tokens buy you the part it is bad at. The price sheet is simply passing that through.

This has an immediate practical consequence that is worth more than most prompt-engineering advice: for cost purposes, a long prompt is cheap and a long response is expensive. An application that retrieves generously and answers concisely is structurally cheaper than one that retrieves narrowly and writes at length, even when the total token count is identical.

Why batch processing is half price

Both providers offer roughly 50% off for work that can wait. The discount is large, uniform, and directly explained by the same physics.

Decode becomes efficient through batching: read the weights once, use them for every sequence in the batch. Arithmetic intensity rises with batch size, and only at large batches does the accelerator stop being memory-starved. But filling a large batch requires enough simultaneous requests, and interactive traffic arrives when it arrives.

A batch API removes that constraint. The provider gains the right to schedule your work whenever the batch is full and the hardware is otherwise idle — which raises utilisation, the term that dominates the cost of running a rack. Half the price is what that scheduling freedom is worth.

The corollary is a genuine architectural opportunity. Any workload without a human waiting — document processing, enrichment, evaluation runs, classification backfills — is a candidate for half-price execution, and the engineering cost of moving it is usually a queue and a callback.

Reasoning tokens bill at the expensive rate

Models that reason before answering produce intermediate tokens, and those tokens are output tokens. They are generated by the same sequential, memory-bound decode process as the visible answer, and they are charged accordingly.

This is where the ratio compounds into something that surprises finance teams. A model that thinks for two thousand tokens before writing a two-hundred-token answer has billed you for two thousand two hundred output tokens. The reasoning is invisible in the response and entirely visible on the invoice.

The spread within a single vendor's reasoning line shows how much this matters. OpenAI lists o3 at $2 input and $8 output per million, and o1-pro at $150 and $600 — a seventy-five-fold difference in price per token across models nominally doing the same category of work, before accounting for how many reasoning tokens each one generates to reach an answer.

The practical discipline is to measure cost per completed task rather than per token. A model that costs five times more per token and solves the problem on the first attempt, without a long reasoning trace, can be cheaper than a cheap model that iterates. The token price is an input to that calculation, not the answer to it.

Geography carries a price

One line on Anthropic's price sheet makes the facilities argument more concretely than any projection: US-only inference is available at 1.1× standard pricing.

Nothing about the model changes. The weights are identical, the architecture is identical, the tokens are identical. What changes is a constraint on which data centres may serve the request — and that constraint costs ten per cent.

That premium is the price of foregoing global load balancing. A provider free to route work anywhere can fill batches from a worldwide pool, follow cheap power around the clock, and use capacity wherever it is idle. Pin the work to one jurisdiction and all three advantages narrow. The surcharge is, quite literally, the cost of geography.

It is worth noting for anyone whose data residency requirements are a preference rather than an obligation: that preference has a measurable price, and it is now printed on the tariff. For anyone whose requirements are genuine — under a regulation such as the EU AI Act or a sectoral rule — it is simply a cost of doing business, and a modest one against the alternative of self-hosting.

The floor beneath the list price

List prices are set by competition, and competition among well-capitalised firms can push prices below cost for a long time. What competition cannot do is change what the compute costs to produce.

That floor has three components. Amortised hardware, which falls per unit of work as each generation improves. Facility overhead, multiplied by PUE — Google reports a comprehensive fleet-wide figure of 1.09 for 2025, against the 1.54 global average it cites from the Uptime Institute's 2025 survey. And energy, which is the term with the least favourable trajectory.

Utilisation ties them together. A provider serving many customers with varied traffic patterns fills batches far more easily than a single organisation serving its own diurnal load, which is the structural reason API pricing beats self-hosting for most workloads until volume gets high and steady. UniverseBlend's breakdown of the hidden fees in a self-hosted bill covers the lines that make that comparison closer than it first appears.

Why this is a facilities question

The energy term is where pricing stops being about silicon and starts being about buildings, substations and contracts.

The International Energy Agency estimates data centre electricity consumption at "around 415 terawatt hours (TWh), or about 1.5% of global electricity consumption in 2024", rising to "around 945 TWh by 2030" — "just under 3% of total global electricity demand". Consumption has grown "by around 12% per year since 2017, more than four times faster than the rate of total electricity consumption".

Within that total, the accelerated portion is what matters for token prices. The IEA projects accelerated server consumption growing "by 30% annually in the Base Case, while conventional server electricity consumption growth is slower at 9% per year", with accelerated servers accounting for "almost half of the net increase".

Supply is the harder half. The IEA's analysis of energy supply for AI puts generation for data centres at "460 TWh in 2024 to over 1 000 TWh in 2030 and 1 300 TWh in 2035" in the Base Case, met in 2024 by a mix of roughly 30% coal, 27% renewables, 26% natural gas and 15% nuclear. Its higher-growth Lift-Off Case reaches "nearly 2 000 TWh by 2035" — and notes that "long grid connection queues mean that most of the additional increase" beyond 2030 is met by fossil fuels.

That clause is the whole argument for treating token pricing as a facilities question. Compute demand can double in eighteen months. A grid connection cannot. When the binding constraint is the queue rather than the chip, the marginal cost of serving an additional token is set by whoever can secure power, where, and on what terms — and that is a slow, geographic, contractual business with nothing in common with the pace of model releases. UniverseBlend's survey of the hidden limits on AI compute is a good companion on the physical constraints behind this.

What this predicts, and what it does not

The asymmetry worth internalising: efficiency gains reach customers quickly, and cost increases reach them slowly.

When a provider improves serving efficiency — better batching, lower precision, a more efficient architecture — the saving is immediate, the marginal cost falls, and competitive pressure passes it through fast. That is the mechanism behind the steady decline in price per unit of capability that has characterised the market.

Cost increases move differently. Power is contracted years ahead, capacity is built to plan, and a provider absorbing a higher marginal cost has strong reasons not to reprice while competitors hold. The pressure accumulates in margins and in what gets built, not in the published rate. So a rising energy term shows up first as capacity constraints, regional availability differences and rationing of the newest hardware — and only much later, if at all, in the list price.

What this does not support is a prediction about where prices go next. Too many terms are moving at once: hardware efficiency improving, model architectures getting cheaper to serve, and energy under pressure. The honest statement is that the floor is rising more slowly than the efficiency gains are falling, and that this has held so far rather than that it must continue.

How to actually reduce your bill

In rough order of effect per unit of engineering effort.

• Shorten outputs. The 5× multiple means a token you do not generate is worth five you do not send. Ask for the answer, not the reasoning, when you do not need the reasoning.
• Move anything asynchronous to batch. Half price for work no human is waiting on.
• Match the model to the task. The published range spans roughly fifty-fold from the cheapest tier to the most expensive; a classification step rarely needs the flagship.
• Cache aggressively. Repeated prefixes are the cheapest tokens available, and many workloads repeat far more than their authors assume.
• Measure your actual input/output distribution before optimising. Most teams are surprised by which side of the ratio dominates their spend.

Self-hosting appears on this list only at sustained high utilisation, and for the reason set out above: a provider fills batches from many customers, and you fill them from one.

Frequently asked questions

Why do output tokens cost more than input tokens?

Because they are produced by a different computation. Input tokens are processed in parallel in a compute-bound phase that uses the accelerator efficiently. Output tokens are generated one at a time in a memory-bound phase, where the hardware must read the model's weights for every token and spends most of its time waiting on memory. The roughly 5× price ratio reflects that asymmetry.

Why is batch processing 50% cheaper?

Because deferring execution lets the provider group your requests into large batches and run them when capacity is otherwise idle. Large batches are the mechanism that makes decode efficient, so the provider's cost per token falls substantially. The discount is payment for scheduling flexibility.

Will token prices keep falling?

Price per unit of capability has fallen consistently, driven by hardware efficiency and architectural improvements. That trend is real but not guaranteed to continue: the energy and facilities floor beneath list prices is under upward pressure from demand growing far faster than grid capacity. No one can responsibly predict the net effect, and anyone who states a figure confidently is guessing.

Is it cheaper to self-host than to use an API?

Only at sustained high utilisation. A provider fills batches from thousands of customers with uncorrelated traffic; you fill them from your own diurnal load. That utilisation gap, plus the overhead of operating a serving stack, is why the crossover point is higher than most estimates assume.

What does electricity actually contribute to a token's cost?

It is a component rather than the dominant one — amortised hardware is larger today. But energy is the term growing fastest and the one most exposed to constraints outside any provider's control, which is why it matters more to the long-run floor than to this quarter's invoice.

Final takeaway

The price list is more informative than it looks. The 5× output multiple tells you decode is memory-bound. The 50% batch discount tells you utilisation is the provider's central problem. Read together, they say that what you are buying is not tokens but scheduled access to memory bandwidth.

Beneath that, the floor is made of things that move on the timescale of infrastructure rather than software — power contracts, substations, connection queues. Model releases arrive every few months; grid connections do not. Any long-run view of what inference will cost has to account for the slower of those two clocks.

Sources and further reading

• Anthropic Claude API pricing, September 2026
• OpenAI API pricing, September 2026
• IEA, Energy and AI — executive summary, data centre consumption and projections
• IEA, Energy and AI — energy supply, generation mix and grid connection queues
• Google data centre efficiency — 2025 fleet-wide PUE of 1.09
• NVIDIA GB200 NVL72 specifications