The Great Equalization
Something remarkable happened in January 2026. Two open source models—DeepSeek-V3.2 and Llama 4 70B—achieved performance parity with GPT-4 on standard benchmarks. Not GPT-3.5. Not Claude 3. GPT-4. This is the inflection point we've been waiting for.
That's the headline, and it's real. What follows is the part the headline skips: where open-weight models are genuinely sufficient, where they aren't, what self-hosting costs once you account for idle GPUs and on-call rotations, and the licensing details that decide whether you can legally ship what you just downloaded. Note the scope — this is about the open-weight ecosystem and running models on infrastructure you control. For the laptop-and-Ollama version, that's a separate guide: Digital Sovereignty: Why Your Next AI Will Live on Your Mac.
DeepSeek-V3.2: The Quiet Giant
DeepSeek (from Chinese AI lab High-Flyer) released V3.2 with little fanfare, and on the public coding and general-knowledge benchmarks it lands in the same band as the previous generation of frontier models — close enough that the ordering flips depending on which leaderboard you read, which evaluation harness produced it, and when the snapshot was taken. Treat any precise figure quoted for a comparison like this, here or anywhere else, as unusable without the harness and the date attached; the number that should decide anything is the one you get from your own task set. What isn't ambiguous is the part that matters commercially: this is a model you can download and run for free.
What's remarkable is the efficiency. DeepSeek-V3.2 uses a Mixture-of-Experts (MoE) architecture with 671B total parameters but only activates 37B per token. This means you get GPT-4 quality on consumer hardware. A single RTX 4090 can run the quantized version at a comfortably usable, conversational pace — no multi-GPU rig or datacenter card required.
Llama 4: Meta's Masterpiece
Meta's Llama 4 doesn't quite match GPT-4 on raw benchmarks, but it has a secret weapon: the ecosystem. A 128K context window, native multimodal support, and first-class support in Hugging Face, Ollama, and LangChain make it the most practical open model for real development.
The Llama 4 8B model is the real story. It's small enough to run comfortably on a modern laptop at conversational speed, and across a large share of everyday development tasks it's hard to tell apart from a hosted mid-tier model — while carrying no per-token bill and no rate limit. Throughput depends heavily on your quantization, your context length, and how much unified memory the machine has, so the number worth having is the one you measure on your own hardware, not the one in someone else's blog post.
When Open Weights Are Enough — and When They Aren't
The useful question isn't "have open models caught up." It's "caught up at what." Capability is not one number, and the gap between open and closed is wildly uneven across task types. Our rough map after a year of running both side by side:
| Workload | Open weights verdict | Why |
|---|---|---|
| Classification, extraction, tagging | Clearly sufficient | High volume, narrow output space, easy to evaluate. This is where self-hosting pays off fastest. |
| Summarization and rewriting | Sufficient | Quality differences exist but rarely change the outcome; cost per call dominates. |
| RAG over your own documents | Sufficient, with caveats | Retrieval quality matters more than model quality here. Watch context degradation on long inputs. |
| Everyday code completion | Sufficient | Short-horizon and heavily patterned. An 8B-class model handles most of it. |
| Multi-file refactors, agentic coding | Still behind | Requires sustained plan-holding across many steps, which is exactly where frontier models pull ahead. |
| Hard one-shot reasoning | Behind | Novel architecture decisions, research-grade math, complex legal analysis. The gap is narrowing but real. |
| Production multimodal (audio, video) | Behind | Open equivalents exist per-modality, but integration quality lags hosted stacks significantly. |
The pattern: open weights win on volume work and lose on judgment work. That's more actionable than any leaderboard, because it maps onto your own request mix. If most of your calls are classification and summarization, you have a self-hosting case regardless of what the frontier does next.
What Self-Hosting Actually Costs
You'll see the claim that local inference costs "$0 after hardware." That's true in the sense that owning a car makes driving free. The honest accounting has three lines, and only one is the GPU.
1. Utilization is the number that decides everything
A hosted API bills per token, so cost tracks usage exactly. Owned or reserved GPU capacity bills for wall-clock time whether or not anyone is using it — a GPU idle at 3am costs the same as one saturated at noon. So the self-hosting math turns almost entirely on utilization: steady round-the-clock volume amortizes beautifully, while a spiky consumer app with a 10x daily peak either over-provisions for that peak or drops requests during it. Plot your request volume by hour for a typical week before committing. If the chart looks like a mountain range rather than a plateau, hosted APIs are probably still cheaper despite the higher headline rate.
2. The operational surface you inherit
Self-hosting means owning what the provider was quietly doing for you: continuous batching so concurrent requests don't serialize, KV-cache management so long conversations don't exhaust memory, autoscaling that copes with model load times measured in tens of seconds rather than milliseconds, and an evaluation suite to catch quality regressions when you change quantization or upgrade a base model. Plus an on-call rotation for an inference tier that is now a hard product dependency. None of it is exotic engineering, but it's a recurring cost measured in engineer-weeks — the line item most self-hosting business cases omit entirely.
3. Where the savings are actually real
None of that is an argument against self-hosting — it's an argument for doing it where the economics are lopsided. Those cases are consistent: high token volume on narrow tasks, agentic loops that revise a draft dozens of times, overnight batch pipelines, and anything where per-token pricing makes a feature structurally unprofitable. Our breakdown of token economics covers how to model this against current API rates.
The Licensing Traps
"Open weights" and "open source" are not synonyms, and conflating them is how legal reviews get ugly late. Specifics vary by model and version, so read the license for the exact release you're deploying — but these are the recurring categories to check:
- Community licenses, not OSI licenses. Meta's Llama releases ship under a bespoke community license rather than Apache or MIT. It permits broad commercial use, but it's a custom contract with conditions — an acceptable-use policy, and a threshold above which very large operators must negotiate separately with Meta. Most companies are unaffected; the point is you're agreeing to terms, not receiving a public-domain grant.
- Attribution and naming requirements. Several licenses require you to state that your product is built with the model, and constrain how derivative models may be named. Cheap to comply with, embarrassing to discover after your marketing site ships.
- Restrictions on training other models. Some licenses restrict using outputs to train a competing model. If your plan is distillation into a smaller in-house model, read that clause first — it's often the whole plan.
- No IP indemnification. Major commercial API providers offer contractual indemnity against copyright claims arising from model output. Open weights generally don't, and training-data provenance isn't auditable. For some legal departments this alone is decisive.
- Origin-based procurement policy. DeepSeek's weights run entirely on your own hardware — but many enterprises have rules about model provenance by jurisdiction, and teams routinely conflate "running DeepSeek weights locally" with "sending data to DeepSeek's hosted API." Document which you're doing before the security review asks.
The Fine-Tuning Advantage
The hard-to-replicate advantage of open weights isn't cost — it's fine-tuning. A model adapted on your own codebase learns your internal libraries, naming conventions, and architectural patterns in a way no amount of prompt engineering reproduces, and on that narrow slice it can beat a much larger general model. Closed APIs offer varying degrees of tuning, but not weight-level control or the right to keep the resulting artifact.
Two caveats before you budget for it. Fine-tuning is an ongoing commitment: your codebase drifts, and a model tuned on last year's conventions confidently recommends patterns you've since abandoned. And it teaches style and vocabulary, not judgment — a tuned 8B model writes code that looks exactly like yours while still being wrong about the hard parts.
The Hardware Tiers
You don't need a data center for the smaller models. Rough guidance by scale:
- Individual workstation: a MacBook Pro with 36GB of unified memory comfortably runs Llama 4 8B — our hardware guide for the agentic era compares Apple silicon against discrete GPUs on this workload.
- Small team: a Linux workstation with a 24GB consumer GPU runs Llama 4 70B quantized. Fine for internal tooling, not for serving customers.
- Production serving: multi-GPU nodes with 48GB or more per card — and here the operational surface above, not the hardware, is the real cost.
- Frontier open weights: Llama 5, at 600B parameters, is a self-hosted-server model, not a laptop model. Don't plan a local-first product around it.
If your reason for self-hosting is regulatory rather than economic, also read up on the cryptographic middle ground in zero-knowledge AI — for some compliance regimes, confidential computation on hosted infrastructure is an easier sell than running your own inference tier.
The Sequel: Llama 5 Changes the Calculus
This piece was written when Llama 4 was the newest open-weight contender. That didn't last. On April 8, 2026, Meta shipped two models the same day: Llama 5, a 600-billion-parameter open-weight model with a 5-million-token context window, and Muse Spark, the first closed model out of Meta Superintelligence Labs, built around native multimodal reasoning. That 5M window lets a self-hosted deployment load an entire monorepo, years of support tickets, or a company's full legal history into one session — impossible with Llama 4's 128K.
Muse Spark, notably, isn't open. That split strategy — open flagship for developers, closed flagship for frontier reasoning — is now the template every major lab is following, and it's the structural reason the open/closed gap persists on the hardest tasks rather than closing entirely.
| Model | Type | Params (active/total) | Context Window | Released |
|---|---|---|---|---|
| DeepSeek-V3.2 | Open weight (MoE) | 37B / 671B | 128K | Jan 2026 |
| Llama 4 70B | Open weight | 70B | 128K | 2025 |
| Llama 5 | Open weight | 600B (total) | 5M tokens | Apr 8, 2026 |
| Muse Spark | Closed (Meta Superintelligence Labs) | Undisclosed | Undisclosed | Apr 8, 2026 |
The Architecture Most Teams Actually Land On
Almost nobody who thinks this through ends up all-open or all-closed. The stable configuration is a router: a cheap self-hosted open-weight model takes the high-volume, easily-evaluated majority of requests, and a frontier hosted model — GPT-6 Astra or a cheaper GPT-5.6 tier from ChatGPT, or Claude Opus 5 (with Claude Fable 5.1 above it) from Claude — takes the minority needing real judgment. Three things make it work:
- Route on task type, not user tier. Routing by customer plan is tempting and produces inconsistent quality for the same feature.
- Build the evaluation set before the router. You cannot decide which requests are safe for the open model without scored examples per task type. This is the step teams skip and then regret.
- Keep an escape hatch both ways. Failing over to the hosted model when your GPU tier degrades — and to the open model during a provider outage or price rise — is most of the resilience argument for a hybrid setup.
The New Normal
The honest 2026 summary: open weights have closed the gap on most production workloads, and with Llama 5 they've pulled ahead on raw context length. What hasn't changed is that "free to download" and "cheap to operate" are different claims. Treat open weights as a serious engineering option with real fixed costs and real license terms, not as a way to make your AI bill disappear — the decision usually makes itself once you've plotted your request volume and read the license. For where the closed frontier sits today, our GPT-5.6 vs Claude Fable 5.1 comparison is the reference point, and best AI coding tools covers the products built on both.