The economics of building, hosting, or calling a model have shifted hard in the past 18 months. Two years ago the default answer was obvious: use the API, move fast, worry about cost later. In 2026, that answer is no longer obvious for a meaningful slice of companies. API costs have restructured, open-weight model quality has caught up with proprietary frontier models for a significant set of tasks, and GPU infrastructure has become more accessible and more competitive.
At the same time, "we're going to train our own model" has become a line I hear from companies that have no business saying it — and "just call the API" has become a cop-out for companies that should have moved to self-hosting eighteen months ago.
Here is the 2026 decision, with actual numbers.
The Three Options, Honestly Described
Before diving into the math, I need to be precise about what each option actually means in 2026, because the terms are often used loosely.
API (buy compute and model): You call a hosted provider — OpenAI, Anthropic, Google, Mistral, Cohere — and pay per token. No infrastructure. No ML team required. Maximum flexibility, maximum per-token cost, total dependency on the provider.
Self-hosted open-weight (buy compute, use free model weights): You run a model like Llama 3.3, Mistral Large, Qwen 2.5, or DeepSeek V3 on your own cloud infrastructure or dedicated GPU servers. You pay for compute, not tokens. Requires ML infrastructure competence. No per-token cost above compute. Full data control.
Fine-tune or train (build the model): You take a base model — either a foundation model or an open-weight model — and adapt it with your own data through fine-tuning, instruction-tuning, RLHF, or full pre-training. The spectrum here is enormous: from a LoRA fine-tune that takes a weekend and costs a few hundred dollars, to a full domain pre-training run that costs millions and takes months.
Most of the confusion in the "build vs. buy" conversation comes from conflating these three options — particularly conflating self-hosting with building, and light fine-tuning with actual model development.
The 2026 Cost Math
Let me give you real numbers at different usage scales. This is based on current pricing as of Q1 2026 and infra costs I work with directly.
At 50M tokens/month (early product stage)
This is typical for a product with a few thousand active users and moderate AI feature density.
| Approach | Monthly Cost | One-Time Setup | Notes |
|---|---|---|---|
| GPT-4o API | ~$12,500 | $0 | At $2.50/1M input + $10/1M output, mixed 70/30 |
| GPT-4o mini API | ~$750 | $0 | At $0.15/$0.60 — often sufficient quality |
| Claude Sonnet 3.7 API | ~$9,000 | $0 | At $3/$15 input/output |
| Self-hosted Llama 3.3 70B (A100 x2) | ~$3,500 | $8,000–15,000 | Infra setup, ongoing ops overhead |
| Self-hosted Mistral Large (2× H100) | ~$6,000 | $10,000–20,000 | Higher VRAM req for larger model |
At 50M tokens/month, the API wins unless you have data sovereignty requirements. The self-hosting overhead — engineering time to deploy, maintain, monitor, and upgrade your serving stack — costs more than the token savings.
At 500M tokens/month (scaling product stage)
This is where the economics start to flip for GPT-4-class tasks.
| Approach | Monthly Cost | Annual Cost | Notes |
|---|---|---|---|
| GPT-4o API | ~$125,000 | ~$1.5M | Linear scaling |
| GPT-4o mini API | ~$7,500 | ~$90,000 | If quality is acceptable |
| Azure OpenAI PTU (provisioned) | ~$70,000–90,000 | ~$900K–1.1M | Predictable latency; commitment required |
| Self-hosted Llama 3.3 70B (H100 x8) | ~$28,000–40,000 | ~$340–480K | Includes cluster ops, monitoring |
| Self-hosted Mistral Large 2 (H100 x16) | ~$55,000–75,000 | ~$660–900K | Larger model, more hardware |
At 500M tokens/month, self-hosting Llama 3.3 70B saves approximately $85,000–97,000 per month versus GPT-4o for tasks where quality is equivalent. That is roughly $1M per year. That is a number that justifies an ML infrastructure hire.
The critical question is not just whether the cost saving is real — it is whether the quality is equivalent. For a significant portion of production workloads, it is.
The quality parity is narrower than you think
Llama 3.3 70B and Mistral Large 2 match or exceed GPT-4o on structured extraction, classification, summarisation, code generation for common languages, and document analysis. They lag on nuanced multi-step reasoning, novel creative synthesis, and tasks requiring broad world knowledge beyond training cutoff. Know which bucket your workload sits in before you make the infrastructure decision.
At 5B tokens/month (AI-native product)
At this scale, self-hosting is not a cost optimisation — it is the only viable business model.
| Approach | Monthly Cost | Notes |
|---|---|---|
| GPT-4o API | ~$1.25M | Economically unviable for most products |
| GPT-4o mini API | ~$75,000 | May be viable depending on margins |
| Self-hosted 70B (H100 cluster x32) | ~$180,000–250,000 | Includes full MLOps and redundancy |
| Fine-tuned smaller model (7–13B) | ~$30,000–60,000 | If task scope is narrow enough |
| Dedicated hardware (owned/collocated) | ~$100,000–150,000 | 12–18mo payback vs. cloud |
At this scale, you need either a fine-tuned smaller model for your specific task or a dedicated GPU infrastructure strategy. The cost structure of public cloud GPUs starts to look as inefficient at token scale as it did for compute-heavy SaaS in 2018.
When Fine-Tuning Makes Sense
Fine-tuning is over-reached for and under-used simultaneously. Teams reach for it when better prompting would solve the problem. And teams avoid it when it is the right answer because they have inflated expectations of the effort required.
A light fine-tune — LoRA or QLoRA on a 7B or 13B base model for a specific task — costs $200–800 in compute, takes 2–5 days end-to-end, and can produce a model that outperforms GPT-4o on narrow tasks at less than 1% of the inference cost. This is genuinely underused.
A full instruction-tune or RLHF run on a 70B base model costs $50,000–200,000 in compute, requires a team with ML training experience, and takes 4–8 weeks. This is appropriate for organisations with high-volume, high-stakes, specialised tasks and the data to train on.
Pre-training from scratch — building a foundation model — costs tens of millions and requires a dedicated ML research team. This is appropriate for organisations that need to own a unique capability that does not exist in any open-weight model: specific domain knowledge, a proprietary corpus, or regulatory requirements that prevent using any third-party model weights. For most companies, this is not the answer.
| Fine-tuning Type | Compute Cost | Time | When to Use |
|---|---|---|---|
| LoRA / QLoRA (7–13B) | $200–800 | 2–5 days | Narrow tasks, high volume, format consistency |
| Full fine-tune (7–13B) | $2,000–8,000 | 1–2 weeks | Better generalisation on specific domain |
| LoRA (70B) | $5,000–20,000 | 1–3 weeks | High-quality narrow tasks at scale |
| Instruction-tune / RLHF (70B) | $50,000–200,000 | 4–8 weeks | Alignment, safety, instruction following |
| Domain pre-training | $1M–10M+ | Months | Proprietary knowledge, regulatory isolation |
Fine-tuning does not fix bad data or vague task definitions
The most common failure mode I see: a team invests in fine-tuning to "make the model smarter" without a precise task definition and a curated training set. The result is a model that is harder to improve than the base model and no better at the actual task. Fine-tuning amplifies the signal in your data. If your data is noisy or your task is underspecified, fine-tuning amplifies the noise.
The Decision Framework
The question I walk clients through is not "build vs. buy" — it is a sequence of five yes/no decisions.
The flow looks like this:
1. Is data sovereignty a hard requirement? If your data cannot leave your jurisdiction or your infrastructure due to regulatory, contractual, or security requirements, the API option is constrained or eliminated. Self-hosting is not optional — it is mandated. This is common in insurance (PII), healthcare, defence, and certain financial services contexts.
2. What is your monthly token volume? Below 100M tokens/month: use the API. Between 100M and 500M: evaluate self-hosting for your highest-volume tasks. Above 500M: self-hosting is almost certainly cheaper and should be evaluated seriously.
3. Does your task sit in the quality parity window? For classification, extraction, summarisation, structured generation, and code assistance on common languages: open-weight models at 70B are broadly equivalent to GPT-4o. For novel reasoning, complex multi-hop synthesis, or tasks requiring broad world knowledge: frontier models still lead. Run a quality benchmark on your actual production task before making the infrastructure decision.
4. Do you have — or can you hire — the ML infrastructure capability? Self-hosting a 70B model in production requires: GPU cluster management, model serving infrastructure (vLLM, TGI, or similar), quantisation decisions, autoscaling, monitoring, and an upgrade path when new weights release. If your engineering team cannot staff this, the cost saving evaporates in ops overhead and incidents. A rough heuristic: you need at least one engineer with ML infra experience for every production model you self-host.
5. Is your task narrow and high-volume enough to justify fine-tuning? If you are doing the same narrow task millions of times per day — entity extraction, document classification, form filling, code review comments in a specific style — a LoRA fine-tune on a 7–13B model may outperform GPT-4o on your specific task at 2–5% of the inference cost. Run the math.
What I Actually See Working
Across the organisations I work with, the pattern that consistently performs in 2026 is a tiered model strategy rather than a single answer.
Most mature AI-native products I work with have converged on a three-tier model architecture:
Tier 1 — Simple, high-volume tasks: A fine-tuned or base open-weight model at 7–13B, self-hosted, for classification, extraction, routing, and intent detection. Cost: less than 1% of API equivalent.
Tier 2 — Standard reasoning and generation: A self-hosted 70B open-weight model (Llama 3.3 or Mistral Large 2) for document generation, analysis, summarisation, and complex Q&A. Cost: 15–25% of GPT-4o API equivalent.
Tier 3 — Frontier tasks: An API call to GPT-4o, Claude Opus, or Gemini Ultra for genuinely complex multi-step reasoning, novel synthesis, and edge cases that require frontier capability. Cost: full API pricing, but volume is small.
The companies that have not built this tiering are either over-spending on frontier API calls for tasks that do not need them, or under-serving their users with a weaker model than the task requires.
The Decision You Are Actually Making
The "build vs. buy" framing is the wrong frame. The decision is: what is the right model capability at the right cost at the right operational complexity for each task your product performs?
That answer changes at different token volumes, changes as open-weight model quality improves (which it is doing rapidly), and changes as your team's ML infrastructure competence matures. It is not a one-time architectural decision — it is a recurring strategic review.
The companies that are getting this right are running that review quarterly. They are benchmarking new open-weight releases against their production tasks. They are measuring cost per feature, not just cost per token. And they are building the model abstraction layer that lets them move between providers and hosting options without a six-month re-engineering effort.
AI platform architecture and build-vs-buy strategy are areas where I work directly with engineering and product leadership. If you are making this decision and want to pressure-test the numbers against your actual workload, let's talk — book a 30-minute discovery call and we will run the analysis together.