Embedding
A dense vector representation of text (or other data) where semantic similarity corresponds to vector proximity.
An embedding is a mathematical representation of meaning encoded as a vector in high-dimensional space. An embedding model maps text (a passage, a phrase, a document) to a fixed-length vector, usually 256 to 1536 dimensions. The key insight is that semantically similar texts produce vectors that are close together in this space — the cosine distance or Euclidean distance between vectors reflects semantic similarity.
How they're made
An embedding model — often a transformer-based neural network — compresses text into a dense vector representation. The model is trained such that similar passages produce similar vectors. Common embedding models include OpenAI's text-embedding-3 (1536 dimensions), Cohere Embed, and open-source models like all-MiniLM-L6-v2 (384 dimensions) and bge-large-en (1024 dimensions).
Common models
Embedding quality and cost vary widely. Fast models like all-MiniLM are suitable for large-scale indexing on a budget. Larger, more expensive models like OpenAI's text-embedding-3-large produce richer vectors. Most RAG systems use embeddings to power similarity search in vector databases.
Where they fit
Embeddings power similarity search in vector DBs and are central to RAG. LLMind does not require embeddings — file-level semantic metadata is structured (human-readable, typed) rather than vectorized, enabling AI agents to read enriched files directly without similarity search.