Open weights vs closed APIs

The big three, and what actually differs and What you actually pay for covered the hosted option. This lesson covers the other one: downloading the model and running it yourself. Before the mechanics, which are in Running a model yourself, you need to know what you are downloading, what you may do with it, and how much quality you give up. Anything here that can date is marked with the month it was read.

1.Open means the weights, and the licence decides the rest

A model, from What a model is made of, is a very large pile of numbers. Those numbers are the weights. When a lab says a model is open, it almost always means you can download that file.

That is useful, and it is much less than open source normally means.

In software terms it is a binary you may modify, not the source. You can run it, quantise it, fine-tune it and serve it. You cannot rebuild it or audit what went into it. So say open weights. A few efforts do publish data and code too, and they say so loudly, because it is rare.

There is no single open-weights licence either. The Hugging Face Hub keeps a separate identifier for each Llama release, from llama2 to llama4 (as of Oct 2026), because the terms changed between them. The licence belongs to the version you downloaded.

Four clauses are worth finding by name every time:

• Commercial use. Allowed at all, and allowed at your size?
• Output usage. May you use the model's answers to train another model, and what does that other model then owe? If you plan to distil, this clause is the plan.
• Redistribution and hosting. May you ship the weights in your product, or serve them to customers?
• Acceptable use. A list of banned applications that you agree to enforce.

Two real examples, as read in Oct 2026. Google's Gemma terms say a model trained on Gemma-generated data counts as a "Model Derivative" when the aim is to make it perform like Gemma, so your small model inherits the terms. The Llama 3.1 licence lets you train on its outputs, but a model you distribute must start its name with "Llama", and above 700 million monthly active users you must ask Meta for a licence. Both would surprise a team that never opened the file.

Open weights is a distribution model, not a licence. The only thing that tells you what you may do is the file that ships with the download.

The blog post announcing a model is marketing. The LICENSE file in the weights repo is the agreement. This is not legal advice; anything with conditions attached goes to someone who gives it.

2.The honest trade, and how big the gap really is

Neither side wins. Which trade suits you depends on facts about your product, not about the models.

The quality row is the one people argue about, so hold it carefully. As of May 2026, Epoch AI measured the best open model trailing the closed frontier by about 4 months on average, or about 8 points on its Epoch Capabilities Index. Under a stricter test, where the open model must score strictly higher, it was about 6 months. In Oct 2025 the same method gave about 3 months. So the lag did not close, and it moves. Epoch is a research nonprofit, but its index is a composite of public benchmarks. It notes that open models tend to do worse on private ones, plausibly because they climb the public ones harder. It also notes that leading closed labs do not always release their best model. Both are reasons the true gap could be larger. In that table the leading open models were mostly from Chinese labs, and their licences differ, so no single rule covers them. I could read only part of the table.

On ordinary work (classify this ticket, extract these fields, summarise this call) the gap is probably smaller than the headline. That is a direction, not a measurement, because Epoch scores aggregate benchmarks and not your task. Epoch's 2025 study of models small enough for home GPUs named distillation, shrinking a big model's behaviour into a small one, as one reason the lag stays short.

Rank is a weaker guide than it looks. A 2025 audit of Chatbot Arena, by researchers at Cohere (itself a model vendor), found open-weight models got far less of the Arena's data and were removed more often. Its data ends in April 2025, and I could not confirm today's practice. Reading a leaderboard covers this.

3.A worked check, a failure, and when to stay hosted

Suppose your team wants a cheap ticket classifier. The plan: have a large open model label ten thousand tickets, then train a small model on those labels. Start with the licence of each candidate, before downloading 15 GB of anything. The Hub exposes it as metadata.

# The declared licence and the gate status, for two candidate repos.
for repo in mistralai/Mistral-7B-Instruct-v0.3 meta-llama/Llama-3.1-8B-Instruct; do
  curl -s "https://huggingface.co/api/models/$repo" \
    | python3 -c "import json,sys; d=json.load(sys.stdin); c=d.get('cardData') or {}; print(d['id'], '| licence:', c.get('license'), '| gated:', d.get('gated'))"
done

When I ran this in Oct 2026 it printed apache-2.0 and False for the first repo, and llama3.1 and manual for the second. The first asks almost nothing of you. The second means you must request access and the owner approves by hand, and then you read its output-usage and naming clauses, because your small model would be a model trained on its outputs. The field is what the repo declares. A re-upload by someone else may carry a different label, so check the original lab's repo, and open the LICENSE file for anything that is not plain Apache-2.0 or MIT.

Now the failure. Picture a team that fine-tuned a hosted model, shipped it, and planned to redo the tuning on each newer base model. On 7 May 2026 OpenAI posted that organisations which had not previously fine-tuned could no longer create fine-tuning jobs. The same notice says that on 6 January 2027 active customers also lose the ability to create new jobs. Existing fine-tuned models keep serving until their base model is retired. Check the deprecations page for the current dates. A capability you rent can change terms, so keep your training data and recipe somewhere you control. With open weights you also keep the file.

The other failure needs no news. A prototype built on a research-only model reaches production unnoticed, and the licence check above would have caught it on day one.

Two things do not apply everywhere. First, the Epoch number is about the best open model, and the best are usually large. Whether one fits the hardware you have is a separate question, answered in What fits on a GPU. Epoch's 2025 study put models that fit one consumer GPU 6 to 12 months behind (as of Aug 2025, on older hardware). Second, the choice is not exclusive. A common, boring design sends the routine majority to open weights in your own infrastructure and escalates the hard remainder to a closed model. Test the split on your own twenty cases, as Picking a model describes, and never on a number from someone else's chart.