Cost Optimisation Guide • 9 min read
How to Reduce LLM Costs: 7 Strategies That Save 45-90%
The most effective LLM cost reduction strategies are prompt caching (90% savings), model downshifting (97% cheaper for simple tasks), and output length constraints (output tokens cost 3× input). Combined, these typically reduce total API spend by 45-90%.
7 Cost Reduction Strategies
#1. Prompt Caching
90% savingsCached prompt tokens cost $0.30/MTok vs $3.00/MTok uncached. Enterprise pipelines re-tokenise the same system prompt across thousands of requests — caching eliminates 90% of that recurring cost.
View research →#2. Model Downshifting
1/30th costStructured prompts enable GPT-3.5 to match GPT-4 on 78% of classification tasks at 1/30th the per-token cost ($0.0005 vs $0.03/1K tokens). Stop defaulting to frontier models for simple tasks.
View research →#3. Output Token Constraints
$11.25/M savedOutput tokens cost 3× input on GPT-4o. Constraining max_tokens from 4,096 to 500 saves $11.25 per million requests. Set explicit length limits in every prompt.
View research →#4. Batch API Processing
50% discountOpenAI's Batch API offers 50% cost reduction ($7.50 vs $15.00/MTok) for jobs completed within 24 hours. Perfect for analysis pipelines, content generation, and data extraction.
View research →#5. Constrained Decoding
0% retriesGrammar-guided generation produces valid JSON on every call — eliminating the 15% retry rate and 2-3× token cost multiplier from failed parses.
View research →#6. Fine-Tuning
80% long-termA fine-tuned GPT-4o-mini eliminates 3,000-token system prompts, saving $1.50/1,000 requests. Over 12 months, this compounds to 80% total prompt cost reduction.
View research →#7. Tiered Model Routing
45% averageRoute 70% of queries to Haiku ($0.25/MTok) and 30% to Opus ($15/MTok). Average cost drops 45% with only 2% quality degradation on complex tasks.
View research →📌 Key Takeaways
- Prompt caching + model routing alone can cut costs by 45-90%.
- Output tokens are your biggest cost lever — always constrain response length.
- Use the ROI Calculator to model savings for your specific workload.
Frequently Asked Questions
How can I reduce LLM costs?
The most impactful strategies are: (1) prompt caching for 90% savings on repeated instructions, (2) model downshifting — using cheaper models for 78% of tasks, (3) constraining output tokens (output costs 3× input), (4) batch API processing for 50% volume discounts, and (5) tiered routing to match task complexity to model cost.
What is the cheapest way to use GPT-4?
Use GPT-4 only for complex tasks that require frontier reasoning. Route 70% of queries to cheaper models like Haiku ($0.25/MTok vs $15/MTok) using complexity-based routing. For batch workloads, OpenAI's Batch API offers 50% cost reduction with 24-hour completion windows.
How much can prompt caching save?
Prompt caching reduces costs by up to 90% on repeated system instructions. Cached prompt tokens cost $0.30/MTok vs $3.00/MTok uncached on Claude 3.5 Sonnet. For enterprise pipelines with thousands of requests sharing the same system prompt, this is typically the single highest-impact optimisation.
Calculate Your LLM Cost Savings
See exactly how much you can save with structured prompts and model routing.
Open ROI Calculator →Cost Reduction: The Evidence
Every claim below is sourced from peer-reviewed research and industry reports.Browse all 141 citations →
Prompt caching reduces static context costs.
Cached prompt tokens cost $0.30/MTok vs $3.00/MTok uncached on Claude 3.5 Sonnet — a 90% reduction on repeated system instructions.
Without prompt caching, enterprise pipelines re-tokenise and re-bill the same system prompt across thousands of requests, paying 10x more for identical static context.
Anthropic, 'Prompt Caching (Beta)' documentation, 2024Model downshifting lowers inference costs.
Structured prompts enable GPT-3.5-class models to match GPT-4 output quality on 78% of classification tasks, at 1/30th the per-token cost ($0.0005 vs $0.03/1K tokens).
Without quality prompts, smaller models produce unusable output, forcing developers to default to expensive frontier models.
Khattab et al., 'DSPy: Compiling Declarative Language Model Calls', Stanford NLP, 2023Output tokens are significantly more expensive than input tokens.
GPT-4o charges $15.00/MTok for output vs $5.00/MTok for input — a 3x premium. Constraining max_tokens from 4096 to 500 saves $11.25 per million requests.
Without output length constraints, LLMs generate verbose responses that consume the most expensive billing vector — output tokens — at 3x the input rate.
OpenAI, 'API Pricing' page, updated 2024Batch APIs drastically reduce high-volume costs.
OpenAI's Batch API offers 50% cost reduction ($7.50 vs $15.00/MTok on GPT-4o output) for jobs completed within a 24-hour window.
Without structured prompt pipelines with deterministic schemas, workloads cannot be batch-processed — every request requires real-time inference at full price.
OpenAI, 'Batch API' documentation, 2024