- TREC 2018
News Track Web Page:
TREC Washington Post document collection
50 test topics for background linking task
relevance judgments for background linking task (with exponential gain values)
The main metric for the background linking task in 2018 was nDCG(5), using exponential gain values. The values map to the following judgments:
Evaluation scores for TREC runs were computed using `trec_eval -q -M1000 qrels run'
- 0: The document provides little or no useful background information.
- 2: The document provides some useful background or contextual information that would help the user understand the broader story context of the target article.
- 4: The document provides significantly useful background ...
- 8: The document provides essential useful background ...
- 16: The document _must_ appear in the sidebar otherwise critical context is missing.
50 test topics for entity ranking task
relevance judgments for entity ranking task