Changing the embedding model underneath a live index

1Why a model swap is not a config change

What an embedding really is covers what an embedding actually is: a list of numbers a model produces such that similar meanings land close together, by whatever notion of "close" that specific model learned during training. That last clause is the part a one-line config change ignores. Closeness is not a property of vectors in general - it is a property of one model's training run. A different model, even one trained on similar data for the same purpose, arranges its space differently. Nothing about the two spaces is required to line up.

So when you re-point your embedding calls at a new model and leave the old vectors sitting in the same index, you don't get worse embeddings. You get numbers from two unrelated coordinate systems being compared as if they were one. The similarity function still runs and still returns a score - it just isn't measuring anything about meaning anymore for any query-document pair that crosses the model boundary.

The last row is what makes this dangerous rather than merely annoying. A schema mismatch throws. A missing file 404s. A stale vector returns a plausible-looking, wrong answer, ranked with total confidence, and the only symptom is that search quality for older content quietly gets worse starting the day of the swap.

A vector is only meaningful next to other vectors from the same model. Treat the model that produced it as part of its identity, not a detail you can change out from under it.

2The migration itself

Because you cannot mix models in one index, migrating means building a second index and moving the corpus into it - never rewriting vectors in place. The pattern has a name in the vector database world: blue-green migration, the same idea used for zero-downtime deploys generally, applied to an index instead of a service.

The steps, in order:

1. Stand up a parallel index configured for the new model's vector dimensions and settings. It starts empty.
2. Turn on dual writes. From this point forward, every new or updated document gets embedded with both models and written to both indexes. This is the step people skip, and skipping it means anything written during the migration window is missing from whichever index you finish second.
3. Backfill the parallel index by walking the entire existing corpus, re-embedding each document with the new model, and writing it in. This is a straightforward batch job, but it is not free - it costs one embedding call per document, on the whole corpus, all at once.
4. Verify before cutting over. Run your retrieval eval set (Measuring retrieval covers building one) against the new index and compare recall@k and MRR against the old index's numbers. A migration that quietly regresses quality is worse than the drift it was meant to fix.
5. Cut traffic over behind an alias, not a hard-coded index name. Your application should query "current-index," and current-index should be a pointer you can move. This turns rollback into flipping the alias back, not re-running the whole migration in reverse.
6. Stop dual writes and retire the old index once you have run confidently on the new one for long enough to trust it - a week of real traffic, not an hour of smoke tests.

# One document's migration record - what to track per vector so you can
# always answer "how much of the corpus is still on the old model?"

from dataclasses import dataclass
from datetime import datetime

@dataclass
class VectorRecord:
    doc_id: str
    embedding_model: str      # e.g. "text-embed-v3", never left implicit
    embedded_at: datetime
    vector: list[float]

def migration_progress(records: list[VectorRecord], target_model: str) -> float:
    """Fraction of the corpus already re-embedded with the target model."""
    done = sum(1 for r in records if r.embedding_model == target_model)
    return done / len(records) if records else 0.0

Tag every vector with the model that produced it and the timestamp, from the very first document you ever index - not just during a migration. Without that tag, "how much of the corpus is still on the old model" is a question you cannot answer, which means you find out the migration is incomplete from a reader's bad search result instead of from a query against your own metadata.

The named-vector approach some databases offer is a faster variant of the same idea: instead of a second collection, you add a second named vector field to the existing one, dual-write into both fields, then switch which field your queries read. It skips copying the rest of the document's metadata, but the underlying discipline - dual write, backfill, verify, cut over behind a pointer - is identical.

3Deciding whether to migrate at all

Not every better-sounding model justifies this. A full re-embed costs one embedding call per document in the corpus, which on a corpus of any real size is a real bill and a real amount of time, and a migration that goes wrong is a production incident. So the decision to migrate deserves its own discipline, separate from the mechanics above.

Evaluate candidate models on a schedule you control - quarterly is a reasonable default - rather than reacting to a vendor's launch post or deprecation notice. And evaluate them on your own retrieval eval set, over your own domain's documents and queries, never on a public leaderboard score alone. A model can top a general benchmark and still underperform your current one on the specific kind of text your corpus is full of - internal jargon, a particular document structure, a language mix a general benchmark doesn't represent. The leaderboard tells you what's worth trying next; it cannot tell you what will actually win on your data.

If the calculus doesn't clear that bar - the gain is real but the corpus is large and the re-embed cost is steep - a middle path exists. A small adapter can be trained to map vectors from the old model's space into an approximation of the new model's space, using a modest set of documents embedded by both, without touching the rest of the index. It recovers most, not all, of the accuracy gain, at a small fraction of the cost of a full re-embed, and it's worth knowing about specifically because it turns "not worth migrating yet" into "worth a cheap partial migration" for corpora where the full job doesn't pencil out.

Whichever way you decide, write it down in the same place you'd write down any other production dependency: which model is live, since when, on what index, and what the last eval run scored. The version tag on the vector answers "what produced this." A short-lived note in your own docs is what answers "why did we choose it," and that answer disappears fast once the person who ran the migration moves to the next thing.