You've run the eval. The new embedding model beats your current one by four points on recall@10 across every slice that matters. You're ready to ship it. Then you remember your production index has forty million vectors, all embedded with the old model, and the two spaces are about as comparable as two random rotations of a coordinate system. Which is to say: not at all.

There's no math that converts old vectors into new-space vectors. You can't blend them in a hybrid index. You can't cosine-compare a query embedding from the new model against a document embedding from the old one and get a number that means anything. If you shipped a partial migration where half your vectors are in each space, your ranking would collapse and you wouldn't get a clean error telling you why.

The pattern that solves this is dual-index with a version column: keep both indexes live, backfill the new one in a throttled queue, shadow-compare live queries during backfill, flag the cutover, and keep the old index around for a week in case rollback is needed. It's the documented standard (Qdrant's docs cover it, several 2026 engineering blogs have written it up), and this issue is for the execution detail rather than the novelty. Issue 014 covered migrating the completion model; this week's issue is the same discipline applied to the embedding layer, where the mechanics are meaner in ways that matter.

Why old and new vectors don't compare

Two embedding models produce vectors in two different coordinate systems. There's no rotation, no linear transform, no cheap trick that converts one to the other. Even if both models produce 1024-dimensional output, "position [0, 0.4, ..., -0.2] in Model A" and "position [0, 0.4, ..., -0.2] in Model B" describe unrelated meanings.

This isn't a caveat you can hedge around. Two documents that were near-neighbours in the old space might sit on opposite sides of the new one, and a query embedded in the new space compared to a document embedded in the old space returns a similarity score that's essentially random. The score won't look wrong; it'll just be a number. Your ranking will silently degrade to somewhere between "worse than a keyword baseline" and "actively hostile to the user".

The consequence for migration is severe. You can't do a rolling replacement where each vector gets re-embedded one at a time and the index stays live in the mean time. Any query during that window that hits a mix of old and new vectors produces noise. You can't do a partial backfill where you re-embed the "important" documents first, because whatever fraction is still in the old space poisons ranking for every query that would have wanted them. And you can't detect the problem from your existing eval, because your eval measures the top-K on a small fixed corpus; the production ranking degrades silently across the long tail.

The one thing you can do is run two indexes in parallel, keep track of which vector lives where, and serve queries only from a self-consistent index at any given moment. That's the dual-index pattern.

The dual-index pattern

Keep the old index. Create a new one, same shape but pointed at the new embedding model. Add a version column to every vector row (or use two separate collections; either works). Every new write goes to both indexes: the ingest pipeline embeds each document twice, once with the old model, once with the new model, and writes both. Every read still goes to the old index while backfill runs.

Backfill is a batch job that reads every document in the corpus, re-embeds it with the new model, and writes the result to the new index against the same document ID. When backfill completes, both indexes contain the entire corpus. That's when you get to think about switching reads.

The diagram shows the two write paths and one read path. Every new document lands in both indexes on ingest, so the two indexes stay in sync from now on. The backfill worker fills in the historical gap on the new index. The read path is controlled by a cutover flag, which stays off until backfill has completed and shadow-compare (below) says the new index is ranking correctly. When you flip the flag, reads move atomically.

Two properties of the pattern matter. The old index stays queryable throughout, so there's no downtime. And there's exactly one active index for reads at any moment; you never serve queries from a mixed state.

Backfill: throttled, versioned, resumable

Backfill is where the cost and the operational risk live, so the execution detail matters more here than anywhere else in the migration.

Throttle to your embedding API's rate limit. Forty million documents at 500 tokens each is twenty billion tokens of embedding cost. At the cheap end of July 2026 pricing (roughly $0.02 per million tokens for OpenAI's small model), that's about $400. At the more expensive end (closer to $0.13 per million), it's about $2,600. Neither number breaks the budget, but running backfill at full API throttle will trip rate limits and cascade into failures on your live ingest pipeline that's sharing the same key. Cap the backfill worker at roughly 30 percent of your API quota, and alert on 429s.

Version every row. If your vector database supports a metadata column on each vector, put the embedding model version there (emb_v: "old" or emb_v: "v2"). If it doesn't, use two separate collections with clear names. The version column matters because backfill will be interrupted at some point (a worker restart, an API outage, a deploy), and when you resume you need to know which documents you've already done.

Make backfill idempotent. Backfill should be a read-embed-write job keyed on document ID. If you re-run it on a document that already has a new-model vector, it should overwrite that vector cleanly, not append a duplicate. Duplicates in a vector index are hard to notice and expensive to clean up.

Estimate the wall clock. At an embedding rate of 1000 requests per second, which is a reasonable steady state for a single hosted API key, 40M documents take about 11 hours if perfectly parallel and closer to a day with real-world overhead. Plan the migration to span a day or two, and staff it accordingly. Don't start the backfill on a Friday afternoon.

Shadow-compare on live queries

While backfill is running, or once it's done and before cutover, you want to check that the new index actually ranks live queries the way your eval said it would. Aggregate benchmark numbers are one thing; the long tail of real queries is another.

Shadow-compare is cheap to build and expensive to skip. For every live query, retrieve the top-K from the old index (as normal) and from the new index (an extra call). Serve the user the old result. Log both result sets keyed by query. A background job compares the two: how often does the top result match, how often does the top-3 overlap, what's the average rank correlation across the top-10.

You're looking for two signals. The two indexes should agree on high-confidence queries most of the time. If the new index disagrees with the old one on 40 percent of top-1 results, either the new model is much better (great) or one of them has a bug (find out which). The disagreements should also shift in the direction your eval predicted. If your eval said the new model wins on identifier queries and loses on paraphrase queries, the shadow-compare should show that same shape on production traffic.

The cost of shadow-compare is a doubled retrieval call during the shadow period. If retrieval is 50 to 300 ms as covered in Issue 007, that's an extra 50 to 300 ms of background work per query, off the critical path. The extra API cost is small enough to ignore for a week or two. The insurance value is not.

Cutover and rollback

When backfill is complete and shadow-compare has run against production traffic for a few days without surprises, you're ready to cut over. The cutover itself is a boolean flip in a feature flag: reads move from the old index to the new one atomically, at the same moment for every host.

Keep the old index warm for a week after cutover. Don't delete it. Don't stop writing to it, either: keep the dual-write path alive so if you need to roll back, the old index isn't stale. The rollback path is the same feature flag flipped the other way.

The reason for a full week is that some regressions only show up on weekly patterns. A tenant with a batch job that runs on Sunday might not exercise the new index until day 6. A user who logs in once a week might hit a query pattern shadow-compare didn't see. A holiday might change the query mix in a way that surfaces a new failure mode. A week of live traffic on the new index, with the old one still warm behind a flag, is the difference between a clean migration and one you write a postmortem about.

After a week without regressions, stop writing to the old index and drop it in the next maintenance window. Reclaim the storage. Update the ingest pipeline to embed only with the new model. Delete the version column (or the second collection). The migration is done.

Common mistakes

Partial backfill without a version column. You start backfilling, get halfway, decide it's fine to start serving from the "mostly new" index. Ranking collapses on the queries that hit the old-vector remainder. There's no error message; the top-K is just wrong. Version every row, and only cut over when backfill is 100 percent done.

Full re-index in place with downtime. You take retrieval down for six hours, run the re-embed, bring it back up on the new model. This works if you can afford six hours of downtime, but you can't afford six hours of downtime, and the rollback path is "restore from a backup that's now stale by six hours plus however long the incident took". Dual-index means no downtime and a rollback that's a flag flip.

Backfill runs at full API throttle. You share the same embedding API key with your live ingest pipeline. Backfill hits 100 percent of the quota, live ingest starts getting 429s, and the incident becomes about your ingest queue backing up rather than the migration itself. Cap backfill at a fraction of your quota, and give it a separate API key if the vendor allows one.

Delete the old index too soon. The new model looked great in shadow-compare, so on day two after cutover you drop the old collection to reclaim storage. On day five, a specific query pattern surfaces a regression, and now rollback means re-embedding forty million documents with the old model. Keep the old index warm for a week minimum, and only after a week without any regressions do you clean up.

Summary

Two embedding spaces don't talk to each other, so an embedding-model migration has to keep the old index live until the new one is complete and validated. The dual-index pattern is the documented standard: write to both indexes, backfill the historical corpus into the new one on a throttled worker, shadow-compare live queries during backfill, cut over with a feature flag, and keep the old index warm for a week. The cost is 2x storage during the migration and a few dollars to a few thousand for the embedding backfill; the insurance value is a rollback path that a full re-index in place doesn't have. Do the migration on a Tuesday, not a Friday.

Production checklist

  • Add a version column (or use two collections) to your vector store so every vector's embedding model is knowable at query time.

  • Wire the ingest pipeline to write to both indexes: embed each new document with both models, write to both, on the same transaction if the store supports it.

  • Stand up a backfill worker that reads the corpus, re-embeds every document with the new model, writes to the new index against the same document ID, and is idempotent on restart.

  • Cap the backfill worker at roughly 30 percent of your embedding API quota. Give it a separate API key if you can, so it can't take live ingest down.

  • Estimate the wall-clock cost of backfill in advance: (document count × avg tokens × price) plus a full day of clock time at typical throughput. Don't start on a Friday.

  • Run shadow-compare on live queries during backfill: retrieve from both indexes, serve the old, log the deltas. Watch top-1 agreement rate and rank correlation.

  • Cut over with a feature flag when backfill completes and shadow-compare has been steady for at least a few days.

  • Keep the old index warm and dual-write live for a full week after cutover. A week of production traffic covers weekly patterns that shadow-compare misses.

  • After a week without regressions, drop the old index in the next maintenance window and simplify the ingest path to a single embed.

  • Add a "current embedding version" line to the observability stack from Issue 004 so operators can see which index is serving reads at a glance.

Further reading