Latent semantic analysis finds meaning in a table of word counts.

Latent semantic analysis is a method from natural language processing and distributional semantics that derives concepts from the relationships between terms and documents. It compresses a large table of word counts into a small number of underlying dimensions. Words and documents that share those dimensions count as related in meaning.

Latent semantic indexing is the application of latent semantic analysis to information retrieval. Documents and queries are compared in the compressed space, beyond shared words alone. The term later acquired a second life in SEO folklore that has little to do with the method.

Singular value decomposition factorises the term-document matrix into three matrices that describe terms, weights and documents. Keeping only the largest weights reduces the matrix while preserving the main patterns of similarity. It is the mathematical core of latent semantic analysis.

Cosine similarity measures how closely two vectors point in the same direction in the reduced space. A value close to one means high similarity. It is how a query is matched to documents after the reduction.

The distributional hypothesis holds that words with similar meanings occur in similar contexts. Latent semantic analysis turns that hypothesis into arithmetic. Meaning is inferred from where words appear together.

A term-document matrix records how often each word occurs in each document, with words as rows and documents as columns. It is the starting material of latent semantic analysis. Every later step works on these counts.

Synonymy means that authors and searchers use different words for the same thing. Latent semantic analysis was designed to bridge that mismatch by relating words that occur in similar contexts. A document can be found by a word it never uses.

If people named things alike, one word per concept would find everything. In five domains, two people chose the same word for the same thing in at most 18 % of cases.

Recall is the share of all relevant documents a search retrieves. Latent semantic analysis raises it by connecting a query to related terms. The gain in recall can cost precision when related is not the same as relevant.

The vector space model represents documents and queries as vectors of term weights and compares them directly. Latent semantic analysis keeps the vectors and reduces their dimensions. It addresses the vector space model's blindness to synonyms.

If reduced dimensions always found more than shared words, both collections would show a gain. The medical abstracts gain 13 %, the information science abstracts gain nothing.

Probabilistic latent semantic analysis, introduced by Thomas Hofmann in 1999, models the same co-occurrence data with a probabilistic latent variable model. It gives the dimensions a statistical interpretation. It fits observed data better than the original algebraic method.

Susan Dumais is one of the inventors of latent semantic indexing and co-author of its founding work at Bell Communications Research. The patent on the method was filed in 1988. Her later research at Microsoft shaped how people and search systems interact.

Scott Deerwester is the lead author of Indexing by Latent Semantic Analysis, published in 1990 in the Journal of the American Society for Information Science. The paper introduced the method to information retrieval. It remains the standard reference.

Gensim is an open-source Python library, created by Radim Řehůřek, that provides scalable implementations of latent semantic indexing and related models. It made the method usable on large collections. Much practical use of the method runs through it.

Cross-language information retrieval can use latent semantic analysis trained on parallel texts in two languages. Documents in both languages then share one reduced space. A query in one language finds documents in the other.

The semantic proximity effect is the finding that people recall words in quicker succession when latent semantic analysis rates them as similar. It links the method's similarity scores to human memory. The mathematics of word counts tracks something in the mind.

The mathematics, and what the compression costs. The usual treatment shows the method applied. It leaves out the low-rank approximation that defines it, the theorem that makes it optimal, the measure of what is lost, the problem of words with several meanings, and the models that came after it. Without them, the reduction looks like meaning gained for free.

A low-rank approximation replaces a matrix with a simpler one of lower rank that stays as close as possible to the original. Latent semantic analysis is exactly such an approximation of the term-document matrix. Every semantic similarity it reports is a property of that approximation.

Truncated singular value decomposition keeps only the largest singular values and their vectors and discards the rest. It is the step that actually reduces the matrix. What is discarded is treated as noise, whether it was noise or a rare meaning.

The Eckart-Young-Mirsky theorem proves that truncated singular value decomposition gives the best possible approximation of a matrix at a given rank. Carl Eckart and Gale Young published it in 1936, and Leon Mirsky extended it in 1960. It is the reason the method is optimal in its own terms, and only in those terms.

The Frobenius norm measures the size of a matrix as the square root of the sum of its squared entries. It quantifies how far the reduced matrix lies from the original. The loss of the compression can be stated as a number.

Singular values are the weights on the diagonal of the decomposition, ordered from largest to smallest. They show how much each latent dimension contributes. Choosing where to cut them decides how many dimensions of meaning survive.

If the number of dimensions kept were a technical detail, every row would score about the same. Around 300 dimensions the model answers three times as many items as with all dimensions kept.

Left singular vectors describe the terms in the latent space, one coordinate per dimension. They show how each word loads on each hidden factor. They are where the semantics of the method can actually be inspected.

Non-negative matrix factorization decomposes a matrix into factors without negative values, as Daniel Lee and Sebastian Seung showed in 1999. Its dimensions are easier to read as parts of a whole. Latent semantic analysis produces negative weights that have no plain meaning.

Latent Dirichlet allocation is a generative probabilistic topic model introduced by David Blei, Andrew Ng and Michael Jordan in 2003. It addressed limits of probabilistic latent semantic analysis. It replaced the method for most topic modelling.

Incremental singular value decomposition updates an existing decomposition when new documents arrive, without recomputing it. It makes the method usable on collections that grow. A static decomposition ages with every new document.

The rank of a matrix is the number of its linearly independent rows or columns. It sets the upper limit for the number of latent dimensions. The chosen number of dimensions is always a small fraction of it.

Explicit semantic analysis, introduced by Evgeniy Gabrilovich and Shaul Markovitch in 2007, represents meaning through human-defined concepts such as encyclopedia articles. Its dimensions have names. It stands against latent dimensions that nobody can label.

Polysemy is the property of a word to carry several related meanings. Latent semantic analysis gives each word one vector, so all its meanings are averaged into one position. The method bridges synonyms well and separates the meanings of one word poorly.

Word2Vec, introduced by Tomas Mikolov and colleagues at Google in 2013, learns word vectors with a shallow neural network from local contexts. It rests on the same distributional hypothesis. It replaced matrix decomposition in many applications, and its vectors are learned from words just the same.

A sparse matrix is a matrix in which most entries are zero. A term-document matrix is extremely sparse, because each document uses only a tiny share of the vocabulary. The sparsity is the reason the reduction is needed.

A dense matrix is a matrix in which most entries are non-zero. The reduced representation of latent semantic analysis is dense. Storage and computation change with it.

The out-of-vocabulary problem arises when a word was absent from the collection the model was built on. Such a word has no position in the space at all. A new term, a new product or a new name is invisible to the model until it is rebuilt.

Random projection reduces dimensions by projecting data onto random directions, justified by the lemma of William Johnson and Joram Lindenstrauss. It approximately preserves distances at far lower cost than a decomposition. It is the fast alternative for very large collections.

Principal component analysis finds the directions of greatest variance in centred data and is computed with the same decomposition. Latent semantic analysis is closely related to it, applied to uncentred word counts. The method belongs to classical statistics as much as to retrieval.

Automated essay scoring rates written answers by comparing them with reference texts. Latent semantic analysis was one of the early methods used for it. The method judged texts by the words they shared with good answers.

Text cohesion is the connectedness of consecutive sentences in a text. Latent semantic analysis has been used to measure it through the similarity of neighbouring sentences. Word overlap and meaningful connection are not the same, and the measure shows both where they meet and where they part.

It shows that semantic similarity can be computed from word counts alone. The meaning the method finds is a compressed pattern of the words that were actually written. Semantic search stands on the lexical level, and latent semantic analysis is the clearest proof of it.

If meaning can be computed from word counts, a model that has only read encyclopedia text should score like human test takers. The two scores differ by a tenth of a point.

The words on a page remain the material from which systems infer meaning. A page that uses the exact terms of its field, and the terms that belong around them, gives any semantic model something to find. The phrase LSI keywords promises a shortcut through that work, and it borrows the name of a method that never offered one. The page LSI keywords, an SEO hallucination explains how the name was borrowed.