Every planning document for an AI product eventually hits the same line item: "cost per token." Teams treat it like a fixed utility bill — something to budget around, not something that moves. That assumption has been wrong for years, and it's getting more wrong every quarter. The unit economics of running a large language model have been falling faster than almost any other input cost in software history, and a roadmap built on today's prices will look conservative, not accurate, within twelve months.
This isn't a claim about any single vendor's discount. It's a structural pattern: the cost of a unit of machine intelligence — one token processed or generated — has been on a multi-year decline driven by several independent forces stacking on top of each other. Understanding why that decline happens, where it's headed, and where it stalls out is now a planning skill, not just a finance one.
What "price per token" actually measures
A token is roughly three-quarters of a word — the atomic unit LLM providers meter and bill on. Every API call has an input token cost (what you send: prompts, documents, conversation history) and an output token cost (what the model generates), usually priced per million tokens and usually different from each other, with output priced several times higher than input because generation requires more compute per token than reading already-processed context.
When people talk about the "price of intelligence" falling, they mean this per-token rate is dropping over time for a comparable level of capability. That's the key qualifier — comparable capability. A model that costs a fraction of what a frontier model cost two years ago, while matching or beating that older frontier model's benchmark scores, is a real price decline. A model that's simply cheap and worse is not the same phenomenon; it's just a different point on the price-quality curve that always existed.
Three separate but overlapping trends get lumped into "token prices are falling":
- Frontier-to-frontier decline: the best available model this year costs less per token than the best available model did in a prior year, for equal or better quality.
- Tier compression: capability that used to require a top-tier, expensive model now runs adequately on a mid-tier or small model at a much lower price point.
- Effective-cost engineering: the same model gets cheaper to use in practice through caching, batching, and request-shaping — the sticker price per token doesn't change, but the bill does.
All three matter for planning, and they call for different responses.
Why the price keeps falling
No single factor explains the trend — it's the compounding of several unrelated ones.
Hardware throughput keeps improving. Each generation of AI accelerators delivers more inference throughput per dollar and per watt than the last. Inference cost is largely a function of FLOPs-per-token divided by hardware efficiency; as the denominator improves, the same model gets cheaper to serve even with no algorithmic change at all.
Algorithmic and architectural efficiency compounds separately from hardware. Techniques like better attention mechanisms, quantization, distillation, and mixture-of-experts routing let providers deliver a given quality bar with fewer active parameters or less compute per token. This is a genuinely different lever from hardware — it's why price-per-quality has fallen faster than hardware cost curves alone would predict.
Competitive pressure across providers forces re-pricing. Once more than one lab can serve a comparable capability tier, price becomes a lever labs pull against each other. A capability that was exclusive and expensive becomes commoditized and cheap once a second or third credible provider can match it.
Specialization reduces waste. Early API usage patterns often ran everything through the most capable (and most expensive) available model because that was the only reliable option. As tiered model families matured — a small, fast model; a balanced mid-tier model; a top-end model for the hardest problems — teams could route routine work to cheaper tiers, effectively lowering their blended cost per unit of useful work even where sticker prices per model stayed flat.
Pricing mechanics multiply the effect. Beyond the headline per-token rate, providers increasingly offer prompt caching (repeated context billed at a fraction of the fresh-read rate), batch processing (non-urgent workloads at a standard discount), and adjustable "effort" or reasoning-depth settings that let a request spend less compute — and cost less — when the task doesn't need maximum depth. These are cost controls layered on top of the base rate, and they compound with it.
The pattern, illustrated by pricing tiers
You don't need historical numbers to see the shape of this — it's visible in a single snapshot of any current model lineup. Providers now ship several tiers simultaneously, each trading capability for cost, and the existence of that ladder is itself evidence of the trend: the tier that would have been the only option a few years ago is now the cheapest option today.
| Tier | Typical use case | Relative input cost | Relative output cost |
|---|---|---|---|
| Frontier / flagship | Hardest reasoning, novel problems, long-horizon agentic work | Highest | Highest |
| Balanced / mid-tier | Most production workloads — coding, analysis, drafting | Moderate | Moderate |
| Fast / small | High-volume, latency-sensitive, simple classification or extraction | Lowest | Lowest |
The mid-tier model in most current lineups now matches or exceeds the flagship model's capability from a couple of generations prior, at a fraction of that older flagship's price. That's tier compression in action, and it's the mechanism that does the most damage to a fixed cost forecast — the assumption "we need the expensive tier for this" quietly stops being true well before anyone revisits the assumption.
Layered on top of the base tiers, several pricing mechanics change the effective price independent of the sticker rate:
| Mechanism | What it does | Typical effect on effective cost |
|---|---|---|
| Prompt caching | Reuses previously-processed context instead of reprocessing it | Cached portions billed around one-tenth of fresh-read rate |
| Batch processing | Processes non-latency-sensitive requests asynchronously | Roughly half of standard synchronous pricing |
| Adjustable reasoning effort | Lets a request use less (or more) internal deliberation | Lower effort settings cost meaningfully less per request for tasks that don't need deep reasoning |
| Fast / priority modes | Trades a price premium for lower latency | Higher cost per token in exchange for faster responses on latency-critical paths |
A team that only tracks the headline per-token rate is missing most of the actual cost-reduction surface available to them. The sticker price is the least controllable lever; caching, batching, effort-tuning, and tier selection are the ones an engineering team can pull today.
Why this matters right now
The reason this deserves attention now rather than being filed under "things get cheaper eventually" is that the decline has been steep enough, and consistent enough across providers, that it changes what's economically rational to build, not just what's technically possible. Capabilities that were cost-prohibitive to run at scale — processing every support ticket with a capable model instead of keyword rules, running a model over every document instead of a sample, giving every user session a persistent reasoning agent instead of a scripted flow — cross the threshold from "interesting demo" to "sane default" as the per-unit cost keeps dropping.
This has a second-order effect that's easy to miss: falling prices don't just make existing workloads cheaper, they change which workloads get attempted at all. A feature that was shelved eighteen months ago because the cost-per-request made the unit economics impossible may already be viable at today's prices — and a team that hasn't revisited the shelved idea is leaving margin, or a shipped feature, on the table.
Practical implications for builders and businesses
The falling-price trend changes how teams should architect systems, not just how they should budget for them.
Design for model substitution, not model attachment. Code that hardcodes a specific model ID throughout a codebase makes it expensive to capture a price or capability improvement later. Abstracting model selection behind a configuration layer — even a simple one — means a price drop or capability jump on a cheaper tier can be adopted in an afternoon instead of a migration project.
Route by task difficulty, not by habit. Not every request needs the most capable model. Classification, extraction, formatting, and routine drafting typically run well on smaller, cheaper tiers; reserve the expensive tier for genuinely hard reasoning, ambiguous judgment calls, or long-horizon agentic work. A tiered routing layer — even a crude one based on task type — often cuts blended cost more than waiting for the next price cut.
Treat caching and batching as default infrastructure, not optimizations. For any workload with repeated context (a long system prompt, a shared document, a stable set of tool definitions) or any workload that isn't latency-sensitive (nightly reports, bulk classification, backfills), the discount from caching and batching is often larger than the discount from waiting for a cheaper model generation. These are available today and don't require betting on future price movement.
Re-evaluate previously-rejected use cases on a fixed cadence. Because tier compression can make yesterday's "too expensive" project into today's "obviously worth it" project, it's worth periodically revisiting a backlog of ideas that were shelved specifically for cost reasons — not ideas rejected for technical infeasibility, but ones rejected on unit economics.
Separate quality requirements from cost defaults in procurement conversations. When a team says "we need the top-tier model," it's worth asking whether that's a measured requirement or an inherited assumption from when the top tier was the only viable option. As the trend continues, that gap between "requires it" and "defaults to it" tends to widen.
A short checklist for teams auditing their own usage:
- Map current requests to task type and check whether each is running on the cheapest tier that reliably meets the quality bar.
- Identify any workload with repeated or shared context that isn't using prompt caching.
- Identify any non-latency-sensitive bulk workload that isn't using batch processing.
- Confirm reasoning-effort or "thinking" settings are tuned per task rather than left at a single default across every request type.
- Set a recurring review (quarterly is reasonable) to re-price the workload against current tiers, not the tiers that were current when the system was built.
Limitations and open questions
The trend is real, but it isn't unconditional, and treating "prices always fall" as a law rather than a pattern creates its own risks.
Output tokens stay comparatively expensive, and long-reasoning modes multiply usage. Models capable of deeper, longer internal reasoning before answering can consume substantially more tokens per request than a shallow response would — a genuine capability gain, but one that can offset or outweigh a headline price cut on a per-request basis if left untuned. Falling per-token price and falling per-task price are not the same measurement, and a team optimizing for the wrong one can see its bill rise even as the rate card falls.
Frontier-tier pricing hasn't collapsed at the same rate as mid-tier pricing. Compression has been most dramatic in the middle of the market, where capability that used to be exclusive to the top tier becomes available lower down. The genuine frontier — the newest, hardest-to-match capability — has not fallen nearly as fast, because it isn't yet subject to the same competitive and commoditization pressure. Planning for cheap frontier-tier access on a fixed timeline is a riskier bet than planning for cheap mid-tier access.
The trend is not guaranteed to continue at the same slope. Past efficiency gains came from a combination of hardware improvements, algorithmic breakthroughs, and competitive dynamics that could plateau, especially if any single factor (chip supply, energy costs, a slowdown in algorithmic innovation) becomes a binding constraint. A roadmap that assumes an indefinite, smooth exponential decline is making a forecasting error different from — but just as real as — assuming no decline at all.
Total cost of ownership includes more than the API bill. Falling token prices lower one input to the total cost of running an AI feature, but latency, reliability, integration engineering, evaluation and monitoring infrastructure, and failure-mode handling don't automatically get cheaper alongside them. A cheaper model that requires more prompt engineering, more guardrails, or more human review to hit the same reliability bar may not be a net cost win.
Price and capability don't always move together for a given task. A cheaper model that's dramatically faster for high-volume, low-complexity work isn't necessarily interchangeable with the model it's replacing for edge cases and ambiguous inputs. Cost-driven model swaps deserve the same evaluation rigor as any other model change, not a pass because the motivation was budgetary rather than capability-driven.
What to watch next
A few signals are worth tracking if you want to stay ahead of this trend rather than react to it after the fact:
- New pricing mechanics beyond the base rate. Effort-based pricing, task-scoped budgets, and speed/priority tiers are relatively recent additions to how providers charge for inference. More granular pricing controls — paying differently for different depths of reasoning, different latency guarantees, or different context-window usage patterns — are a likely next step, and they change the calculus of "what's the cheapest way to accomplish this task" independent of the sticker price per token.
- Continued tier compression at the frontier. Watch whether capability that's currently frontier-exclusive starts appearing in mid-tier models on a similar cadence to what's happened at lower capability levels. That would be the strongest signal that the compression pattern is holding rather than slowing.
- Convergence of specialized and general-purpose pricing. As models get better at routing internally — deciding for themselves how much computation a given sub-task deserves — the distinction between "the cheap model" and "the expensive model" may blur into a single model that prices itself dynamically per request based on difficulty.
- How competitors respond to each other's price moves. Because much of this trend is competitive rather than purely technical, a price cut from one provider is a leading indicator for cuts from others, typically within a similar release cycle.
FAQ
Why do LLM token prices keep falling?
A combination of better AI accelerator hardware, more efficient model architectures (quantization, distillation, sparse routing), and competitive pressure between providers. Each factor would lower costs on its own; together they compound into a faster decline than any single trend would produce.
Is the price drop the same at every model tier?
No. Compression has been strongest in the middle of the market, where capability once exclusive to top-tier models becomes available in smaller, cheaper models. Genuine frontier capability — the newest and hardest-to-match tier — has fallen more slowly because it faces less competitive pressure.
Does a lower price per token always mean lower total cost?
Not automatically. Reasoning-heavy or long-output tasks can consume far more tokens per request, which can offset a lower per-token rate. Total cost per completed task, not per token, is the metric that matters for planning.
What's the difference between prompt caching and batch processing?
Prompt caching reduces cost for reused or repeated context within requests (billing the repeated portion at a fraction of the standard rate). Batch processing reduces cost for non-urgent, asynchronous workloads by trading immediate response for a lower flat rate. They solve different problems and are often used together.
Should I always use the cheapest available model?
No. Cost should follow a task-difficulty assessment, not the other way around. Route routine, well-defined tasks to cheaper tiers and reserve the most capable (and most expensive) tier for genuinely hard reasoning or judgment calls, then validate that the cheaper tier actually meets your quality bar before switching.
How often should a team re-evaluate its model and pricing choices?
A quarterly review is a reasonable cadence given how fast tiers and pricing mechanics change. The review should check both the sticker price of the tiers in use and whether caching, batching, and effort settings are still tuned to current workload patterns.
Will token prices eventually hit zero?
Unlikely in any near-term sense — inference still requires real compute, energy, and hardware, all of which have costs. The more realistic trajectory is continued decline in price-per-unit-of-capability, with genuinely new frontier capability continuing to carry a premium even as older capability tiers keep getting cheaper.
Teams that want help auditing their current model usage or architecting a cost-aware inference layer can reach out to Woyce Technologies.