Why Data Teams Should Run Gemini Embeddings on Vertex AI

Your retrieval pipeline probably has three different embedding vendors bolted together: one for the original prototype, one someone swapped in because it scored better on a benchmark, one nobody remembers choosing. Every one of them means a separate API key, a separate rate limit to babysit, and a separate outage to explain to your platform team. If your data already lives in Google Cloud, that's three problems you don't need.

gemini-embedding-001 is now generally available in both the Gemini API and Vertex AI, and for a data team already running BigQuery, Cloud Storage or Vertex AI Search, it's the harder case to argue against, not for.

One vendor, one IAM boundary, one bill

Every embedding call from a separate provider is an egress path, a credential to rotate, and a line item your security team has to review on its own. Route gemini-embedding-001 through Vertex AI instead and it's just another model call: same service account, same VPC Service Controls perimeter, same Cloud Billing export your finance team already parses. When an FDE hands a RAG pipeline over to a customer's platform team, "one fewer vendor to onboard" is worth more than a fraction of a percentage point on a benchmark.

It also means your embedding calls sit next to your generation calls in the same quota dashboard, so a traffic spike doesn't quietly exhaust a rate limit on a provider nobody on the on-call rotation has credentials for.

Gemini embeddings on Vertex AI are the current quality bar

This isn't just a convenience trade. gemini-embedding-001 currently sits at number one on the MTEB Multilingual leaderboard, the standard benchmark suite for retrieval, classification and clustering quality. It's also the model Google is actively pointing customers toward: text-embedding-004, text-embedding-005 and text-multilingual-embedding-002 are deprecated and scheduled to retire on January 14, 2026, so teams still on the old models are migrating whether they planned to or not.

The model is trained with Matryoshka Representation Learning, which means you can truncate its native 3072-dimension output down to 1536 or 768 dimensions and keep most of the retrieval quality, at a fraction of the storage and query cost in your vector index. That's a real lever for cost, not a marketing footnote: halving your embedding dimension roughly halves your Vertex AI Vector Search or pgvector storage bill.

import vertexai
from vertexai.language_models import TextEmbeddingModel, TextEmbeddingInput

vertexai.init(project="<your-project-id>", location="us-central1")
model = TextEmbeddingModel.from_pretrained("gemini-embedding-001")

doc_input = TextEmbeddingInput(
    text="Refund policy: customers can return unused items within 30 days.",
    task_type="RETRIEVAL_DOCUMENT",
)
embedding = model.get_embeddings([doc_input], output_dimensionality=768)[0]
vector = embedding.values  # list[float], length 768

The task_type argument matters as much as the model choice. Set it to RETRIEVAL_DOCUMENT when indexing content and RETRIEVAL_QUERY when embedding a user's search, and the model optimizes the vector space for that asymmetry instead of treating both sides identically. Other supported values, SEMANTIC_SIMILARITY, CLASSIFICATION, CLUSTERING, QUESTION_ANSWERING and FACT_VERIFICATION, cover the other shapes of pipeline a data team actually builds.

The honest objection: online calls take one text at a time

Here's the part that will annoy anyone reindexing a large corpus. Older Vertex AI embedding models accepted up to 250 texts per online prediction request. gemini-embedding-001 accepts exactly one. If your ingestion job was built around batching 200 chunks per call, that code breaks, not gracefully, the moment you swap the model name.

The fix is to stop treating bulk reindexing as an online workload. Vertex AI's batch prediction API takes a JSONL file of content and task_type pairs from Cloud Storage or BigQuery and processes it asynchronously, which is what large backfills should be using anyway instead of hammering a synchronous endpoint in a loop:

{"content": "Refund policy: customers can return unused items within 30 days.", "task_type": "RETRIEVAL_DOCUMENT"}
{"content": "Shipping takes 3 to 5 business days for domestic orders.", "task_type": "RETRIEVAL_DOCUMENT"}

Submit that file as a batch prediction job and Vertex AI handles the throughput and retries for you. It's a real rewrite if you leaned on synchronous batching before, budget a day for it, not an afternoon.

Where to start

Don't do a big-bang cutover. Point one retrieval collection, ideally a low-traffic one, at gemini-embedding-001 with output_dimensionality=768, compare retrieval quality against your current provider on a sample of real queries, and only then plan the batch reindex for everything else. With the legacy Vertex AI models retiring in January, that experiment is worth running this quarter rather than the week before the deprecation date.

For an FDE, fewer vendors in the retrieval path means fewer things that can page you at 2 a.m., and that's usually worth more than a leaderboard ranking on its own.