The Essence of Vector Embeddings: High-Dimensional Cosine Similarity Explained
In Retrieval-Augmented Generation (RAG) and vector databases (Pinecone, Qdrant, pgvector), text is converted into high-dimensional numerical vectors (e.g., 1,536 dimensions for OpenAI’s text-embedding-3-small). To compare how semantically related two paragraphs are, we calculate their Cosine Similarity. But why cosine similarity instead of Euclidean distance? What actually happens in 1,536 dimensions? 1. The Geometry of Angle vs. Distance Suppose we have two vectors, $\mathbf{A}$ and $\mathbf{B}$, in $n$-dimensional Euclidean space: $$\mathbf{A} = [a_1, a_2, \dots, a_n], \quad \mathbf{B} = [b_1, b_2, \dots, b_n]$$ ...