Contents 4
Sufficient consensus
Question answering returns a direct answer rather than a list of documents, extracting it from passages or generating it from retrieved material. It powers answer boxes, assistants and conversational search.
It is now the default for factual queries.
If search questions had one right answer each, nearly all of them would sit in the first row. Fewer than half do.
| Distinct plausible answers | Development set, 2,002 questions | Test set, 2,004 questions |
|---|---|---|
| One | 49 % | 44 % |
| Two | 23 % | 24 % |
| Three | 14 % | 16 % |
| Four or more | 13 % | 16 % |
Min, S., Michael, J., Hajishirzi, H. and Zettlemoyer, L., AmbigQA: Answering ambiguous open-domain questions, EMNLP 2020. Questions from real Google search queries, answers searched in English Wikipedia.
What the result pages leave out
These belong to the subject. People ask about them. They are missing from the agreed coverage.
- It depends. Many real answers are conditional. The format has no room for the condition, so the condition is dropped.
- The source vanishes. Answers arrive without the context that would let a reader judge them, and often without a link at all.
- Questions nobody asked. The system answers what was typed. The more useful question is frequently the one the person did not know to ask.
- Disagreement flattened. Where credible sources differ, one version is selected and presented as settled.
- Publishing into a system that does not link back. Being the source of an answer that generates no visit is now a normal outcome and it changes what publishing is for.
What people actually want to know
- What does this answer depend on?
- Who says so, and what would change their mind?
- What should I have asked instead?
- If a machine answers for me, what does my page still do?
Write the condition next to the answer. A conditional answer survives being quoted. An unconditional one becomes wrong somewhere.