Word Vectors

Word vectors or word embeddings are numerical representations of words in a multidimensional space. Learning methods assign vectors according to how words occur in training data, so words used in similar contexts often lie closer together. Methods such as Word2Vec popularised these continuous representations and enabled computational similarity comparisons.

Traditional word vectors generally assign the same vector to a word regardless of its sentence and therefore represent ambiguity only to a limited extent. Contextual language models instead generate representations according to the surrounding text. Distances are not objective measures of meaning or truth: they reflect data, objective, and model and may inherit social biases. Search systems now commonly use embeddings for complete passages rather than isolated words.

  • Numerical representation of words
  • Similar usage can produce spatial proximity
  • Word2Vec is a well-known learning method
  • Contextual models account for surrounding text
  • Vector similarity is not a measure of truth

Frequently asked questions

Word vectors are learned numerical sequences that represent words in a vector space and enable computational comparisons.

They represent statistical usage patterns from training data. This is useful for similarity but does not amount to complete human language understanding.
Zum Glossar