2018 News Track

Return to the TREC home page TREC home Return to the TREC
Data home page Data home          National Institute of Standards and Technology Home Page

TREC 2018
News Track Web Page:
TREC Washington Post document collection

50 test topics for background linking task
relevance judgments for background linking task (with exponential gain values)

The main metric for the background linking task in 2018 was nDCG(5), using exponential gain values. The values map to the following judgments:

  • 0: The document provides little or no useful background information.
  • 2: The document provides some useful background or contextual information that would help the user understand the broader story context of the target article.
  • 4: The document provides significantly useful background ...
  • 8: The document provides essential useful background ...
  • 16: The document _must_ appear in the sidebar otherwise critical context is missing.
Evaluation scores for TREC runs were computed using `trec_eval -q -M1000 qrels run'

50 test topics for entity ranking task
relevance judgments for entity ranking task

Last updated: Thursday, 11-Apr-2019 08:00:21 EDT
Date created: April 9, 2019