Changing the embedding model underneath a live index

1.Why a model swap is not a config change

What an embedding really is showed that a model arranges vectors so similar meanings land close together, by whatever notion of "close" it learned in training. Closeness belongs to one model's training run, not to vectors in general. A different model, even one trained on similar data, arranges its space differently, and nothing requires the two to line up.

So if you point your embedding calls at a new model and leave the old vectors in the index, you get numbers from two unrelated coordinate systems compared as if they were one. The similarity function still runs and still returns a score. It just measures nothing for any pair that crosses the model boundary.

The half-migrated index. Take 2 million chunks and a backfill that re-embeds them in place, overwriting each vector as it goes. Every query is embedded by one model, so each choice of query model leaves some of the index in the wrong space. The numbers are invented, to show the shape.

Read the middle row twice. Whichever model embeds the queries, a large share of the corpus is compared across two spaces. A schema mismatch throws, but a mixed index returns five plausible results with total confidence. The only symptom is that search quietly gets worse for a slice of your content.

The drop-in that was not. A same-size vector makes this worse, because the store accepts the write. Voyage AI's June 2026 announcement (vendor-reported) calls voyage-context-4 (as of June 2026) "a drop-in replacement" for voyage-context-3, and its docs, as read in October 2026, list the same output sizes for both. That describes your code, not your stored vectors. Neither page says the two models' vectors can be compared, so assume they cannot. A vendor can promise otherwise: Voyage said in January 2026 that its 4 series produces compatible embeddings. Look for that in writing, a vendor statement and not a hunch.

A vector is only meaningful next to other vectors from the same model. Treat the model that produced it as part of its identity, not a detail you can change out from under it.

2.The migration: a second index, one partition at a time

Migrating means building a second index and moving the corpus across, never rewriting in place. The pattern is blue-green, the idea behind zero-downtime deploys, applied to an index. Qdrant's migration tutorial (as of October 2026, tested on version 1.19.0) gives two versions: a second collection, or a second named vector in the same collection. The discipline is the same in both.

The steps, in order:

1. Stand up a parallel index with the new model's dimension and distance metric. It starts empty.
2. Turn on dual writes. Every new or changed chunk is embedded by both models and written to both. Check what that covers. Qdrant's tutorial covers upserts only, so deletes and partial updates must be paused or replayed, or a chunk deleted mid-migration lingers in the new index.
3. Backfill from stored text, not by re-parsing sources, and write insert-only so a stale backfill write never overwrites a fresher dual write (Qdrant offers that mode from version 1.16). A contextualized model reads whole documents (see Giving each chunk its context), so keep the source text too, as From source system to corpus advises.
4. Verify one partition at a time. Run the eval set from Measuring retrieval against the new index and compare recall@k and MRR (mean reciprocal rank, defined in that lesson) with the old. Fix the bar before you look, and read the queries that got worse, not only the average.
5. Cut over by partition, behind a pointer. Keep dual writes for a week of real traffic, then stop them, snapshot, and retire the old index.

A partition is anything you can route on its own, such as a tenant. No query may span partitions on different models, so a tenant is a safe unit and a topic is not. The router picks both the index and the model that embeds the query. Notion's February 2026 case study describes moving one generation of its indexes at a time, checking correctness before the next. It changed the embedding model and the vector store in the same project, and the post does not split its reported gains between the two. Change one thing per migration, or you cannot say which one helped.

# What to track per vector, so you can always answer
# "how much of this partition is still on the old model?"
import os
from dataclasses import dataclass
from datetime import datetime

# Set to the model you are migrating to. Never hard-code a model id.
TARGET_MODEL = os.environ["EMBED_MODEL"]

@dataclass
class VectorRecord:
    chunk_id: str
    partition: str          # the unit you cut over, such as a tenant
    embedding_model: str    # never left implicit
    embedded_at: datetime
    vector: list[float]

def progress(records: list[VectorRecord], partition: str) -> float:
    """Share of one partition already embedded with the target model.
    In a real store this is a filtered count query."""
    rows = [r for r in records if r.partition == partition]
    done = sum(r.embedding_model == TARGET_MODEL for r in rows)
    return done / len(rows) if rows else 0.0

def ready_to_cut(old_recall: float, new_recall: float, max_drop: float = 0.0) -> bool:
    """Gate: the new index may not score below the old on this partition."""
    return new_recall >= old_recall - max_drop

Tag every vector with its model and a timestamp from the first document you index, not only during a migration. Without the tag, "how much is still on the old model" cannot be asked, and you learn the migration is incomplete from a reader's bad search result.

3.Deciding whether to migrate at all

Run the arithmetic first. Take the same 2 million chunks at about 300 tokens each: 600 million tokens to embed. At an assumed 200 chunks a second (measure yours), that is 10,000 seconds, nearly three hours of pure embedding, longer under rate limits. Storage is 1,024 numbers at 4 bytes, about 8.2 GB per index, so about 16.4 GB while both exist. For money, look up your provider's price per token. A model change also makes every stored hash stale, so the savings in Keeping the index fresh without paying twice do not apply. This is the one time you pay for the whole corpus.

Evaluate candidates on a schedule you control, such as quarterly, not when a launch post lands. A leaderboard tells you what to try next, not what wins on your jargon and layout.

A cheaper middle path. One paper (September 2025, a single author, the author's own tests) trains a small adapter that maps the new model's query vectors into the old model's space, so the old index keeps working. Trained on 20,000 paired old and new embeddings, 2 percent of a 1 million item corpus, it kept about 94 to 99 percent of the new model's recall on three text datasets and one image-model upgrade. On a far bigger jump, from word vectors to a sentence model, it kept about 72 percent. It needs the old model to stay runnable, and the author says a full re-embed is deferred, not avoided. Treat it as a bridge, and test it on your own eval set.

When not to migrate. Skip it when the gain is inside the noise of a 30-query eval set, as Choosing an embedding model explains. Skip it when the corpus is small and rebuilt nightly anyway, since the rebuild is then your migration. Skip it when a vendor guarantees one embedding space across the models you would swap. And wait if you cannot afford two indexes at once.

Whatever you decide, write down which model is live, since when, on which index, and what the last eval scored. The tag on the vector says what produced it, and the note says why you chose it. Once the model is settled, the next bill is keeping the index fed: Keeping the index fresh without paying twice.