- TREC 2018
-
News Track Web Page:
trec-news.org
TREC Washington Post document collection
50 test topics for background linking task
relevance judgments for background linking task (with exponential gain values)
The main metric for the background linking task in 2018 was nDCG(5), using exponential gain values. The values map to the following judgments:
- 0: The document provides little or no useful background information.
- 2: The document provides some useful background or contextual information that would help the user understand the broader story context of the target article.
- 4: The document provides significantly useful background ...
- 8: The document provides essential useful background ...
- 16: The document _must_ appear in the sidebar otherwise critical context is missing.
Evaluation scores for TREC runs were computed using `trec_eval -q -M1000 qrels run'
50 test topics for entity ranking task
relevance judgments for entity ranking task
|
|