Word2Vec (Mikolov et al., 2013, Google) trains by either predicting a word from its context (CBOW) or predicting context from a word (Skip-gram). GloVe (Pennington et al., 2014, Stanford) factorizes the word co-occurrence matrix. Both produce similar results: 100–300 dimensional vectors where cosine similarity correlates with semantic similarity. These vectors capture remarkable relationships: countries map to capitals, verbs map to tenses, and analogies are solvable through vector arithmetic.
Word2Vec and GloVe produce one vector per word, regardless of context. "Bank" gets the same embedding whether it means "river bank" or "financial bank." Contextual embeddings (ELMo, then BERT) solved this by producing different representations depending on context. Modern sentence embeddings (from models like BGE, E5) go further, embedding entire sentences into vectors. Each generation improved on the last, but the core idea — meaning as a vector — started with Word2Vec.
Word2Vec's biggest contribution wasn't the algorithm but the demonstration that neural networks can learn useful representations of language from raw text. This proof of concept inspired the progression from word vectors to sentence vectors to contextual embeddings to full language models. The embedding layer of every LLM is a direct descendant of word embeddings: a lookup table mapping discrete tokens to learned continuous vectors, just at a much larger scale.