Embeddings

An embedding is a list of numbers, typically hundreds or thousands of dimensions, that represents a piece of data, a word, sentence, image, or product, in a way that captures its meaning rather than its raw form. A model learns to place items with similar meaning near each other in this vector space, so "restaurant" and "café" end up closer together than "restaurant" and "spreadsheet."

Embeddings are produced by a trained model, not hand-designed; older approaches used word-level embeddings, while modern systems typically embed whole sentences or documents at once. They are stored and searched in a vector database, where similarity between vectors, usually cosine similarity, stands in for similarity of meaning. This differs from one-hot or keyword representations, which capture exact matches but nothing about relatedness.

Embeddings underpin semantic search, recommendation, clustering of similar documents, and the retrieval step in retrieval-augmented generation systems. A common pitfall is assuming embeddings are stable and comparable across models or versions: vectors from two different embedding models are generally not compatible, and re-embedding a corpus is required whenever the model changes.