Benefits of Smaller Embedding Dimensions for Vector Storage

Your vector column is probably the fattest thing in your Postgres instance, and it got that way by accident: someone picked a default model and nobody questioned the number on the box. Embedding dimensions drive vector storage, index size and memory almost linearly, so trimming them is the cheapest cost lever in a retrieval stack. Truncating Matryoshka-style embeddings, storing half-precision halfvec and binary quantization each shrink it in pgvector, and a recall check tells you what it cost.

What embedding dimensions cost in vector storage

The pgvector reference documents the size of each vector: 4 * dimensions + 8 bytes for vector, 2 * dimensions + 8 for halfvec and dimensions / 8 + 8 for bit. For a million chunks that works out to:

| Column type | Bytes per vector | 1M vectors | | --- | --- | --- | | vector(3072) | 12,296 | about 12.3 GB | | halfvec(3072) | 6,152 | about 6.2 GB | | vector(1024) | 4,104 | about 4.1 GB | | halfvec(1024) | 2,056 | about 2.1 GB | | bit(1024) | 136 | about 0.14 GB |

These are raw vector bytes. Row overhead and indexes come on top.

Benefits of smaller embedding dimensions

1. Smaller tables and indexes

Moving from vector(3072) to halfvec(1024) takes a million-chunk table from about 12.3 GB to about 2.1 GB, roughly 6x. An HNSW index keeps its own copy of every vector, so the saving shows up twice, and an index that fits in RAM is the one that stays fast.

2. Matryoshka truncation keeps most of the quality

Matryoshka embeddings come from models trained so the first N numbers of the vector still work on their own. OpenAI says both text-embedding-3 models were trained this way, and that a 256-dimension text-embedding-3-large vector still beats an unshortened 1536-dimension text-embedding-ada-002 on MTEB, per its embeddings guide. Ask for the size you want when you embed:

from openai import OpenAI

client = OpenAI()

resp = client.embeddings.create(
    model="text-embedding-3-large",
    input="How do I rotate the API key for the billing service?",
    dimensions=1024,
)
vector = resp.data[0].embedding  # 1024 floats

If you cut vectors you already stored, re-normalize them, because truncation breaks unit length. pgvector 0.7.0 and later does both steps in one expression:

INSERT INTO chunks_1024 (id, body, embedding)
SELECT id, body, l2_normalize(subvector(embedding, 1, 1024))::halfvec(1024)
FROM chunks;

Only do this with models built for it. Chopping the vector of an ordinary model can wreck it.

3. halfvec halves the bytes and raises the index ceiling

halfvec stores 16-bit floats, so it is exactly half the size of vector. It also lifts the indexing limit: pgvector indexes vector up to 2,000 dimensions but halfvec up to 4,000. A 3072-dimension vector column can't get an HNSW index directly, yet it can through a cast.

CREATE TABLE chunks_1024 (
  id        bigint PRIMARY KEY,
  body      text NOT NULL,
  embedding halfvec(1024) NOT NULL
);
CREATE INDEX ON chunks_1024 USING hnsw (embedding halfvec_cosine_ops);

-- Or keep the full-precision column and index a cast of it:
CREATE INDEX ON chunks USING hnsw ((embedding::halfvec(3072)) halfvec_cosine_ops);

With the cast version, queries must repeat the expression (ORDER BY embedding::halfvec(3072) <=> ...) or Postgres won't use the index.

4. Binary quantization cuts index storage 32x

binary_quantize turns each dimension into one bit, so a 1024-dimension vector goes from 4,104 bytes as vector to 136 as bit. In Hugging Face's tests, mxbai-embed-large-v1 kept 96.45% of its float32 retrieval score at 32x less storage when the top results were rescored with the full vectors. The same article saw e5-base-v2 fall to 74.77%, so this one depends on the model.

CREATE INDEX ON chunks_1024
USING hnsw ((binary_quantize(embedding)::bit(1024)) bit_hamming_ops);

SET hnsw.ef_search = 100;

SELECT id, body FROM (
  SELECT id, body, embedding
  FROM chunks_1024
  ORDER BY binary_quantize(embedding)::bit(1024) <~> binary_quantize('<query-vector>'::halfvec(1024))
  LIMIT 100
) AS candidates
ORDER BY embedding <=> '<query-vector>'::halfvec(1024)
LIMIT 10;

The inner query finds 100 candidates by Hamming distance on bits, and the outer one re-sorts them with the full vectors. The table still holds those vectors, so what shrinks is the index and the memory it needs. Keep hnsw.ef_search (default 40) at least as high as the inner LIMIT, or the index can return fewer candidates than you asked for.

Check recall on a golden query set

Every percentage above is someone else's data. Yours comes from a golden query set: a few dozen real questions, each with the chunk ids that should come back. Run every variant and compare recall@10.

import json

import psycopg
from openai import OpenAI

K = 10
client = OpenAI()

VARIANTS = {
    "float32-3072 (exact scan)": (3072, """
        SELECT id FROM chunks
        ORDER BY embedding <=> %(v)s::vector(3072) LIMIT %(k)s"""),
    "halfvec-1024 (hnsw)": (1024, """
        SELECT id FROM chunks_1024
        ORDER BY embedding <=> %(v)s::halfvec(1024) LIMIT %(k)s"""),
    "binary-1024 + rerank": (1024, """
        SELECT id FROM (
            SELECT id, embedding FROM chunks_1024
            ORDER BY binary_quantize(embedding)::bit(1024)
                     <~> binary_quantize(%(v)s::halfvec(1024))
            LIMIT 100
        ) AS c
        ORDER BY embedding <=> %(v)s::halfvec(1024) LIMIT %(k)s"""),
}


def embed(text: str, dims: int) -> str:
    resp = client.embeddings.create(
        model="text-embedding-3-large", input=text, dimensions=dims
    )
    return str(resp.data[0].embedding)  # pgvector parses '[0.1, 0.2, ...]'


with open("golden.json") as f:
    golden = json.load(f)  # [{"query": "...", "relevant_ids": [12, 87]}, ...]

with psycopg.connect("postgresql://<user>:<password>@<host>/<db>") as conn:
    conn.execute("SET hnsw.ef_search = 100")
    for name, (dims, sql) in VARIANTS.items():
        found = possible = 0
        for case in golden:
            rows = conn.execute(sql, {"v": embed(case["query"], dims), "k": K})
            returned = {r[0] for r in rows}
            relevant = set(case["relevant_ids"])
            found += len(returned & relevant)
            possible += min(K, len(relevant))
        print(f"{name:28} recall@{K} = {found / possible:.3f}")

The first variant is an exact scan, so it is your ceiling. Ship the smallest variant whose recall stays within a drop you agreed on beforehand, say two points, and raise hnsw.ef_search or the rerank LIMIT before giving up on one that misses.

When smaller embedding dimensions are not worth it

• Small corpora. Fifty thousand chunks at vector(3072) is about 615 MB of raw vectors, a rounding error on the bill, and one recall regression costs more than the gigabytes saved.
• Recall-critical retrieval. If a missed document is the incident (legal discovery, safety, compliance), stay at full precision until the golden set says otherwise.
• Models not built for it. No Matryoshka training means no safe truncation, and some models degrade badly under binary quantization.

In production the vector store is a bill that grows with every document, and a recall number from your own golden set lets you shrink it without guessing.