Contents 4
Sufficient consensus
Distributed epistemology studies knowledge held across groups, artefacts and institutions rather than in single minds. Division of cognitive labour makes complex work possible and makes verification collective.
Search engines are now part of that distribution.
If what gets written down reflected what was found, the two columns would look alike.
| What became of the study | Null results | Strong results |
|---|---|---|
| Never written up | 64.6 % | 4.4 % |
| Written up, never published | 14.6 % | 34.1 % |
| Published outside top journals | 10.4 % | 38.5 % |
| Published in a top journal | 10.4 % | 23.1 % |
Franco, A., Malhotra, N. and Simonovits, G., Publication bias in the social sciences: Unlocking the file drawer, Science, 2014. 221 peer-reviewed survey experiments run through the NSF programme TESS, each with its full publication record.
What the result pages leave out
These belong to the subject. People ask about them. They are missing from the agreed coverage.
- Tacit knowledge. Skill that experienced people cannot fully articulate never enters the index and is therefore never in a result.
- Knowledge inside organizations. The most accurate account of how something actually works usually exists as an internal document nobody will publish.
- Who had a reason to write. Publication follows funding, obligation and marketing. The resulting record is a map of incentives.
- Failures go unpublished. Approaches that did not work are the most useful information and the least documented.
- Asking a person. For many questions the fastest reliable route is a conversation, and search has trained people out of considering it.
What people actually want to know
- Who actually does this work, and has anyone asked them?
- What did the people who tried this and stopped find out?
- Which part of this can only be learned by doing it?
- Who had a reason to publish what I am reading?
Journalistic research and market research outside the web produce material that no competitor can copy from the result page, because it was never on the result page.