searchresultoptimization.com
Start/Information retrieval/Mean average precision

Evaluation metrics measure agreement with a judge. The judge was hired by the system.

A score of 0.87 tells you how well the ranking matched a list somebody wrote down.

Sufficient consensus

Mean average precision averages precision across recall levels and across queries. With NDCG and MRR it is the standard way to compare retrieval systems on a benchmark.

It made the field comparable and it is the reason progress can be claimed at all.

What the result pages leave out

These belong to the subject. People ask about them. They are missing from the agreed coverage.

  • The relevance judgement. Every score rests on a human decision about what counts as relevant for a query. That decision carries the values of whoever made it.
  • Unjudged documents. Documents nobody assessed count as irrelevant. A genuinely new answer is scored as noise.
  • Averages hide the failures. A high mean can contain queries where the system returns nothing useful at all, and those are the queries people remember.
  • What the user wanted to achieve. No standard metric contains the outcome the person needed. It measures the list, not the life.
  • Benchmarks age. Systems are tuned until the test set is exhausted, and the score keeps rising after the improvement has stopped.

What people actually want to know

  • Relevant to whom, decided by whom?
  • Which queries does this system fail completely, and are they mine?
  • Does a better score mean I will find better answers?
  • What is not in the test set?

A metric is a shortened definition of good. Reading the definition is more informative than reading the number.

More in information retrieval