Machine Learning

In Retrieval-Augmented Generation (RAG) and vector databases (Pinecone, Qdrant, pgvector), text is converted into high-dimensional numerical vectors (e.g., 1,536 dimensions for OpenAI’s text-embedding-3-small). To compare how semantically related two paragraphs are, we calculate their Cosine Similarity. But why cosine similarity instead of Euclidean distance? What actually happens in 1,536 dimensions? 1. The Geometry of Angle vs. Distance Suppose we have two vectors, $\mathbf{A}$ and $\mathbf{B}$, in $n$-dimensional Euclidean space: $$\mathbf{A} = [a_1, a_2, \dots, a_n], \quad \mathbf{B} = [b_1, b_2, \dots, b_n]$$ ...

Large Language Models (LLMs) like GPT-4, Claude, and LLaMA appear to understand reasoning, humor, code, and nuance. Yet under the hood, every LLM operates on a single core objective: given a sequence of tokens, predict the probability distribution for the very next token. In this guide, we demystify how LLMs transform human language into math, process context through the Transformer architecture, and generate coherent text. You can interact with the live demos below to see the math in action. ...