Why Switching Embedding Models Means Re-Indexing Everything

Swapping the embedding model name in your config is a one-line change. Switching embedding models safely is not, because vectors from the old and new model can't be compared, so every vector in your index has to be recomputed. Skip that and you get either a loud error or a quiet one, and the quiet one is the dangerous one.

This is the case for planning the re-embed before you need it, plus the four moves that make it boring: tag, build beside, compare, cut over.

What goes wrong when switching embedding models

Different dimensions fail loudly. pgvector rejects a wrong-sized write to a fixed-size column with errors like expected 1024 dimensions, not 256, and a cross-size comparison with different vector dimensions 1024 and 384. Qdrant returns Wrong input: Vector dimension error: expected dim: 256, got 3072. Annoying, but you notice within minutes.

Same dimensions fail quietly. OpenAI's text-embedding-ada-002 and text-embedding-3-small both produce 1536 numbers per text. Query an index built by one with the other and nothing errors: cosine similarity returns a score, the sort runs, and you get ten plausible-looking, mostly wrong results. Each model learns its own coordinate system, so position 40 in one means nothing in the other. Matching length is a coincidence of the format.

A half-migrated index is the worst case. Old and new vectors sit side by side, queries match only one generation, and recall drops with no alert to tell you.

Tag every vector with the model that made it

If rows don't record their model, you can't tell which are stale and you can't run two generations at once. The pgvector README shows the pattern: leave the column without a fixed size, store a model id, and build an expression plus partial index per model.

CREATE TABLE embedding_models (
  model_id bigint PRIMARY KEY,
  name     text NOT NULL,  -- e.g. '<provider>/<model>'
  dims     int  NOT NULL
);

CREATE TABLE embeddings (
  model_id  bigint NOT NULL REFERENCES embedding_models,
  item_id   bigint NOT NULL,
  embedding vector,  -- no fixed size, so two models can coexist
  PRIMARY KEY (model_id, item_id)
);

CREATE INDEX CONCURRENTLY ON embeddings
  USING hnsw ((embedding::vector(1536)) vector_cosine_ops) WHERE (model_id = 1);
CREATE INDEX CONCURRENTLY ON embeddings
  USING hnsw ((embedding::vector(1024)) vector_cosine_ops) WHERE (model_id = 2);

-- query the new generation
SELECT item_id
FROM embeddings
WHERE model_id = 2
ORDER BY embedding::vector(1024) <=> '[<1024 floats>]'
LIMIT 10;

Each index covers one model's rows. The query must repeat the same cast and the same WHERE filter, or Postgres won't use the index. CONCURRENTLY keeps writes unblocked while it builds.

Qdrant fixes vector size and distance when a collection is created, so there the tag is the collection itself. Name collections after the model generation, such as docs_v2, and point an alias at the live one.

Build the new blue-green vector index beside the old one

Never re-embed in place. Create a second index, keep serving from the first, and fill the second in the background. Qdrant's migration tutorial lays out the sequence:

1. Create the new collection with the new model's size and distance.
2. Turn on dual writes, so every new document is embedded twice and stored in both.
3. Backfill old documents in batches with insert-only upserts, so the backfill can't overwrite a newer dual write. The tutorial lists Qdrant 1.16 or later for that mode.

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")

client.create_collection(
    collection_name="docs_v2",
    vectors_config=models.VectorParams(
        size=1024,  # output size of the NEW model
        distance=models.Distance.COSINE,
    ),
)

The tutorial warns that this works as-is only for upserts. Pause deletes and partial updates during the migration, or apply them to both indexes.

Compare retrieval quality on a fixed query set

A new model is better on someone's benchmark, not necessarily on your documents. Pull about 50 real queries from your logs and write down which documents should come back. Embed each query with the model that matches the index you are searching.

labeled = [  # (query, ids of documents that should be returned)
    ("reset sso password", {"doc-18", "doc-204"}),
    ("export audit log", {"doc-77"}),
]

def recall_at_10(search, labeled):
    total = 0.0
    for query, relevant in labeled:
        returned = set(search(query, 10))  # list of document ids
        total += len(returned & relevant) / len(relevant)
    return total / len(labeled)

print("old", recall_at_10(search_old, labeled))
print("new", recall_at_10(search_new, labeled))

The tutorial also suggests MRR and nDCG@10, and says to decide your pass bar before you look. Averages hide the group of queries that got worse, so read the worst regressions by hand.

Cut over with an alias, then keep the old index

The switch should be one atomic step. In Qdrant, delete and recreate the alias in a single request:

client.update_collection_aliases(
    change_aliases_operations=[
        models.DeleteAliasOperation(
            delete_alias=models.DeleteAlias(alias_name="prod")
        ),
        models.CreateAliasOperation(
            create_alias=models.CreateAlias(
                collection_name="docs_v2", alias_name="prod"
            )
        ),
    ]
)

Deploy the new query-embedding model in the same release, or you will search the new index with old-model queries. In pgvector, the cutover is changing which model_id your queries filter on. Keep dual writes running through an observation period: once they stop, rolling back means losing every write since.

The objection: re-embedding is expensive

Sometimes, but check the arithmetic. At $0.02 per million tokens, the listed price for text-embedding-3-small, a billion tokens of text costs $20 to embed. A self-hosted model trades that for GPU hours.

The real costs are engineering time, double storage during the overlap, and the eval set. They are smaller than an unplanned re-embed forced by a deprecation notice, on an index where nobody knows which rows came from which model.

Also, not every new model deserves the move. If the compare step shows no gain on your queries, you just saved the cutover. And if your corpus is small and search can go offline for an hour, drop and rebuild is fine, as long as you still tag the rows.

First step: add the model tag today

Add a model_id (or collection per generation) and backfill every existing row with the model you use now. Then write your 50 queries down. Both take an afternoon, and the next model swap becomes a scheduled job instead of an incident.

In production, an embedding model is a schema, and changing it without a migration plan is how search gets quietly worse while every dashboard stays green.