Contents 4
Sufficient consensus
Word sense disambiguation determines which meaning of an ambiguous term applies in a given context, using surrounding words, entities and knowledge bases. It affects retrieval, translation and answer selection.
Modern systems do it well most of the time.
If systems handled every meaning of a word equally well, the two columns would match. When the word carries a less common meaning, the score falls by 41 to 62 points.
Explicit identifiers settle which of several same-named things a page means. See Entity Disambiguation.
| System | F1, most frequent sense | F1, less frequent senses |
|---|---|---|
| Always the first dictionary sense | 100.0 | 0.0 |
| BERT-base | 94.9 | 37.0 |
| EWISE | 93.5 | 31.2 |
| Gloss-informed bi-encoder (BEM) | 94.1 | 52.6 |
Blevins, T. and Zettlemoyer, L., Moving down the long tail of word sense disambiguation with gloss informed bi-encoders, ACL 2020. English all-words benchmark, first dictionary sense from WordNet.
What the result pages leave out
These belong to the subject. People ask about them. They are missing from the agreed coverage.
- Most of the time is not all of the time. The cases that fail are the specialized ones, which is to say the professionally important ones.
- The dominant sense wins. A term with one common and one technical meaning resolves towards the common one, taking the technical audience away.
- Context you did not supply. A page that never names the field it belongs to leaves the decision to the surrounding vocabulary.
- Cross-language collisions. A term that means one thing in German and another in English produces results that look broken and are working exactly as designed.
- The reader disambiguates too. A heading that could be read two ways is read the wrong way by a share of the audience, and they leave without telling you.
What people actually want to know
- Which meaning of my main term does the system assume?
- Does my page state, early, what field it belongs to?
- Could my headline be read in a way I did not intend?
- Am I competing against an unrelated industry for my own word?
Say which subject you are in, in the first paragraph, in plain words. That single sentence resolves most ambiguity for machines and for readers.