Sufficient consensus
An inverted index maps each term to the documents containing it, which makes retrieval fast enough to be interactive. Crawling, parsing and stemming fill it, and a page has to be reachable, parseable and worth storing to enter it.
This is the floor under every search result.
What the result pages leave out
These belong to the subject. People ask about them. They are missing from the agreed coverage.
- The unindexed majority. Practice knowledge, internal documents, conversations and unwritten experience are not in any index, and searchers treat the index as the world.
- Rendering. Content that only appears after client-side execution may never reach the index at all, which turns a technical decision into an editorial one.
- Selection before ranking. Deciding what to store is a stronger filter than deciding what to rank, and it is invisible to everyone outside.
- Age and removal. Documents leave the index. Links that worked last year now return nothing, and the record of what was written disappears with them.
- The cost of storage as a filter. Making it cheap for a system to store and present your content is a real advantage, and it is rarely described as content work.
What people actually want to know
- Is my content actually in the index, or only on my server?
- What does a crawler see when it renders my page?
- What do I know that is written down nowhere?
- Which of my old pages still exist, and who still links to them?
Result pages can only display existing content. Everything else is a decision somebody made about what was worth keeping.
More in information retrieval
Learning to rankThe system is trained on the behaviour it caused.Snippet generationIt is written by a machine, from your text, for a query you did not know about.Query expansionYou are answering a question that was quietly edited before it reached you.Relevance feedbackAbsence produces no signal.Query performance predictionA page of ten results looks the same whether the answer exists or not.Passage retrievalYour page is being read in fragments by a system that never sees the whole.Mean average precisionA score of 0.87 tells you how well the ranking matched a list somebody wrote down.Vector space modelIf the web never described your case, the vector for it is somebody else's.