Embedding Models

An embedding model processes a piece of text and outputs a fixed-length vector — typically between 384 and 3,072 dimensions depending on the model architecture. Each dimension encodes some aspect of meaning learned during training on large text corpora. The key insight is that these dimensions are not hand-crafted categories like 'topic' or 'sentiment' but emergent properties that the model discovers by observing patterns across billions of sentences.

Because the model is trained to place semantically similar inputs near each other in vector space, the distance between two vectors becomes a proxy for meaning similarity. The phrases 'cancel my subscription' and 'I want to stop my membership' might produce vectors with a cosine similarity of 0.92, while 'cancel my subscription' and 'the weather forecast looks clear' might score 0.12. This geometric property is what makes embeddings useful for search, clustering, classification, and recommendation tasks — all of which fundamentally involve measuring how similar two pieces of text are.

Different embedding models optimize for different properties. Some prioritize cross-lingual alignment, mapping 'thank you' and 'merci' to nearby vectors so that multilingual search works without translation. Others focus on code semantics, placing functionally equivalent code snippets near each other regardless of variable naming. Still others are tuned for specific domains like legal or medical text, where ordinary words carry specialized meanings. The choice of embedding model directly affects downstream task performance, often more than any other single architectural decision — which is why model selection criteria should account for embedding quality alongside generation capability.

Keyword search, including established algorithms like TF-IDF and BM25, works by matching exact terms or their morphological stems between a query and a document. It is fast, well-understood, predictable, and effective when users know the right terminology. Its fundamental limitation is the vocabulary mismatch problem: a search for 'refund complaint' will miss documents that say 'reimbursement issue' or 'money back problem' unless synonym lists are painstakingly maintained by hand.

Embedding-based search — often called semantic search or vector search — sidesteps the vocabulary mismatch problem entirely, as this architecture deep-dive demonstrates. Because all text is mapped to the same continuous vector space during indexing, the system retrieves documents by meaning rather than surface form. This is particularly valuable in enterprise knowledge bases, support ticket routing, and retrieval-augmented generation pipelines where the user's query and the source documents rarely use identical phrasing.

In practice, many production search systems combine both approaches through a technique called hybrid search or reciprocal rank fusion. Keyword search handles exact entity matches, product names, and technical identifiers where precision matters. Embedding search catches conceptually related results that keyword matching misses. The combination consistently outperforms either approach used alone, which is why modern search architectures increasingly treat embeddings and keywords as complementary rather than competing strategies.

The choice of embedding model depends on the use case, the domain, and the deployment constraints. Key factors include vector dimensionality (which affects storage costs and query latency at scale), the quality of the training data relative to your domain, multilingual support requirements, and maximum input length. A model trained primarily on web text and social media may significantly underperform on scientific papers, legal contracts, or internal company documentation that uses specialized terminology.

Public benchmarks like MTEB (Massive Text Embedding Benchmark) provide a useful starting point for model benchmarking for narrowing the field, but they measure performance on standardized academic datasets that may not reflect your data distribution. The most reliable evaluation approach is to build a small test set of query-document pairs drawn from your actual use case — perhaps 50 to 100 examples — and measure retrieval accuracy directly. A model that ranks second on MTEB might rank first on your specific corpus, and the performance gap between models is often larger on domain-specific data than on general benchmarks.

Embedding models also differ in how they handle long text. Some models truncate input beyond 512 tokens, silently discarding content that might contain the most relevant information. Others support 8,192 tokens or more, enabling document-level embeddings without truncation. Understanding your model's context window is essential and connects directly to chunking strategies — the approach you use for breaking long documents into embeddable segments must account for the model's maximum input length.

Cost and infrastructure are practical constraints that often narrow the choice further. Cloud-hosted embedding APIs charge per token and add network latency. Self-hosted models eliminate per-request costs but require GPU infrastructure. For high-volume applications processing millions of documents, the cost difference between a large and small embedding model can be substantial over time.

Embeddings struggle with negation and precise quantitative reasoning. The sentences 'this product is not waterproof' and 'this product is waterproof' often map to very similar vectors because the model learned they appear in similar contexts and discuss the same topic. This is a fundamental limitation of distributional semantics — meaning derived from context patterns cannot reliably capture logical operators like negation. For applications where negation matters (compliance checking, fact verification), embedding search alone is insufficient and must be supplemented with explicit logical reasoning.

Domain drift is another common failure mode. An embedding model trained on general web text will not produce meaningful vectors for highly specialized terminology — chemical compound names, internal company acronyms, niche industry jargon, or regulatory codes. The vectors for unfamiliar terms are essentially random, producing unpredictable and unreliable similarity scores. Fine-tuning on domain-specific data or choosing a domain-adapted model addresses this, but requires an evaluation pipeline to verify that the improvement is real rather than assumed.

Finally, embedding quality degrades silently. Unlike keyword search, where you can inspect the matched terms and understand why a result was returned, embedding search returns results based on opaque distance calculations in high-dimensional space. A retrieval pipeline that worked well last quarter may be producing increasingly poor results because the source documents have changed, the query distribution has shifted, or the embedding model's weaknesses happen to align with the new data. Teams should invest in observability and tracing for their retrieval pipelines to detect relevance degradation before users notice — or before the downstream language model starts generating answers from irrelevant context.