Zubnet AILearnWiki › Cosine Similarity
Fundamentals

Cosine Similarity

Cosine Distance, Vector Similarity
A measure of similarity between two vectors based on the angle between them, ignoring their magnitude. Cosine similarity of 1 means the vectors point in the same direction (identical meaning). 0 means they're perpendicular (unrelated). -1 means opposite directions. It's the standard similarity metric for comparing text embeddings in semantic search, RAG, and recommendation systems.

Why it matters

Every time you do semantic search, use RAG, or compare embeddings, cosine similarity is (probably) the metric deciding what's "similar." Understanding it helps you debug retrieval quality, choose between cosine and alternatives (dot product, Euclidean distance), and understand why some searches miss obvious matches.

Deep Dive

The formula: cos(θ) = (A · B) / (||A|| × ||B||), where A · B is the dot product and ||A||, ||B|| are the vector magnitudes (lengths). By dividing by magnitudes, cosine similarity measures direction only — a vector [1, 2, 3] is identical in cosine similarity to [2, 4, 6] because they point the same way. This normalization is why cosine works well for embeddings: the direction encodes meaning, while magnitude can vary based on text length or model quirks.

Cosine vs. Dot Product

If embeddings are already normalized to unit length (magnitude 1), cosine similarity equals the dot product — and dot product is faster to compute (no division). Most embedding models output normalized vectors for exactly this reason. When using a vector database, check whether your embeddings are normalized: if yes, use dot product (faster). If not, use cosine similarity (correct regardless of normalization).

Limitations

Cosine similarity treats all dimensions equally, but some embedding dimensions may be more important than others. It also measures overall direction similarity, which can miss nuanced differences. Two sentences about "Python programming" and "Python the snake" might have moderately high cosine similarity because they share the "Python" concept. More sophisticated similarity measures (learned metrics, cross-encoder reranking) can capture finer distinctions at higher computational cost.

Related Concepts

← All Terms
← Corpus Cross-Attention →