A practical framework for choosing an LLM for an MVP: evaluate options on four axes: capability, cost per request, latency, and reliability/fallback support. It recommends starting with a mid-tier hosted model, adding a premium fallback for hard prompts, routing simple tasks to cheaper models, and tracking quality and cost weekly with an eval suite.
Picking an LLM for an MVP should be a product decision, not a hype decision.
Evaluate on four axes
- Capability: does the task really need a frontier model?
- Cost: what is acceptable cost per request at your target volume?
- Latency: what p95 response time is acceptable for your UX?
- Reliability: do you have fallback providers for outages and rate limits?
Recommended approach
- Start with a mid-tier hosted model for speed.
- Add a premium fallback for hard prompts.
- Route simple tasks to cheaper models.
- Track quality and cost weekly with an eval suite.
For most MVPs, this gives the best tradeoff between shipping velocity and long-term cost control.
What the Price Lists Look Like in October 2026
Model names change every few months, but the price structure is stable: a small tier, a mid tier and a large tier, with a hundredfold spread between the cheapest and the dearest. These rows come from the providers' own pricing pages, read on October 10, 2026, in USD per million tokens for short-context text requests.
| Tier | Example model | Input | Output |
|---|---|---|---|
| Small | gpt-6-luna (OpenAI) | $0.10 | $0.50 |
| Small | Claude Haiku 5.5 (Anthropic, prompts up to 100,000 tokens) | $0.10 | $0.50 |
| Small | Gemini 2.5 Flash-Lite (Google) | $0.10 | $0.40 |
| Mid | gpt-6-sol (OpenAI) | $2.00 | $10.00 |
| Mid | Claude Sonnet 5.5 (Anthropic) | $2 | $10 |
| Large | gpt-6-astra (OpenAI) | $10.00 | $50.00 |
| Large | Claude Fable 5.1 (Anthropic) | $10 | $50 |
Sources: OpenAI API pricing, Anthropic pricing and Gemini API pricing. The full table, with more models, is in our guide to AI tools and APIs for app development.
Compare on Cost per Correct Answer, Not Cost per Call
A cheaper model that fails more often can still cost less per correct answer, and a dearer model can cost more without being right more often. The method is one division: cost per call divided by the share of answers that pass your evaluation.
For a call with 2,000 input tokens and 500 output tokens, the small tier costs $0.00045, the mid tier $0.009 and the large tier $0.045. If the small tier passes 70% of your cases, it costs about $0.00064 per correct answer. If the mid tier passes 90%, it costs about $0.01, and if the large tier passes 95%, about $0.047. The pass rates here are placeholders to show the method, so replace them with your own measurements.
The cost of a wrong answer decides the rest. A wrong draft that a person reviews is cheap, while a wrong answer that triggers an action is not, and the second case can justify the dearer model even when the first would not.
Build the Evaluation Before You Pick
- Collect real inputs from users, support tickets or documents, not invented examples.
- Write the expected output or the pass rule for each input.
- Run every candidate model on the same set and record pass rate, latency and cost.
- Pick the cheapest model that clears your threshold, and keep the set to catch regressions when a provider updates a model.
Grow the set from production failures. The set you start with is rarely the set you finish with.
Questions That Rule a Model In or Out
- Data handling: does the provider train on your API inputs, how long does it retain them, and in which region are they processed? Read the provider's current terms rather than a summary.
- Context window: does the task need whole documents in one request, or can retrieval keep prompts short?
- Structured output and tool calling: does the model return valid JSON and call tools reliably on your own prompts?
- Rate limits: do the quotas at your launch volume leave room for a spike?
- Change policy: how much notice does the provider give before it changes a price or retires a model?
- Lock-in: can you switch provider by changing configuration, or is the prompt written around one model's quirks?
Route by Task and Keep a Second Provider
Route by task rather than by habit: the small tier handles classification and extraction, the mid tier handles drafting and tool calls, and the large tier is called only when a cheaper model's answer fails a check. Keep a second provider configured so that an outage or a rate limit degrades the service instead of stopping it.
For the cost of running the result, see AI MVP development cost and cloud cost optimization for AI MVPs. If you would rather have the choice made and built for you, see AI MVP development.