TREC Tracks

A TREC workshop consists of a set tracks, areas of focus in which particular retrieval tasks are defined. The tracks serve several purposes. First, tracks act as incubators for new research areas: the first running of a track often defines what the problem really is, and a track creates the necessary infrastructure (test collections, evaluation methodology, etc.) to support research on its task. The tracks also demonstrate the robustness of core retrieval technology in that the same techniques are frequently appropriate for a variety of tasks. Finally, the tracks make TREC attractive to a broader community by providing tasks that match the research interests of more groups.

Each track has a mailing list. The primary purpose of the mailing list is to discuss the details of the track's tasks in the current TREC. However, a track mailing list also serves as a place to discuss general methodological issues related to the track's retrieval tasks. Further, some tracks have track-specific web pages that provide history and background material regarding the track's focus. Thus, this page lists contact information for all the TREC tracks, whether or not the track is scheduled to be run in the current TREC. TREC track mailing lists are open to all; you need not participate in TREC to join a list. Most lists do require that you become a member of the list before you can send a message to it.

The set of tracks that will be run in a given year of TREC is determined by the TREC program committee. The committee has established a procedure for proposing new tracks.

• Blog Track
  The purpose of the blog track is to explore information seeking behavior in the blogosphere.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe trec-blog <FirstName> <LastName>
  
  
• Chemical IR Track
  The goal of the chemical IR track is to develop and evaluate technology for large scale search in chemical documents including academic papers and patents to better meet the needs of professional searchers: specifically patent searchers and chemists.
  Mailing list: follow the link on the web page to join the list
  Chemical IR Track web page
  
  
• Entity Track
  The overall aim of this new track is to perform entity-related search on Web data. These search tasks (such as finding entities and properties of entities) address common information needs that are not that well modeled as ad hoc document search.
  Mailing list: visit http://groups.google.com/group/trec -entity to apply for membership.
  Entity Track web page
  
  
• Legal Track
  The goal of the legal track is to develop search technology that meets the needs of lawyers to engage in effective discovery in digital document collections.
  Mailing list: contact oard (at) umd.edu to be added to the list.
  Legal Track web page
  
  
• Million Query Track
  The goal of the "million query" track is to test the hypothesis that a test collection built from very many very incompletely judged topics is a better tool than a collection built using traditional TREC pooling.
  Mailing list: Follow the instructions given on the track web page to join the email list million (at) cs.umass.edu
  Million Query Track web page
  
  
• Relevance Feedback Track
  A new track in 2008. The goal of the relevance feedback track is to provide a framework for exploring the effects of different factors on the success of relevance feedback.
  Mailing list: Follow the instructions given on the track web page to join the email list.
  Relevance Feedback Track web page
  
  
• Web Track
  The Web track explores Web-specific retrieval tasks, including diversity and efficiency tasks, over collections of up to 1 billion Web pages. The 2009 Web Track will be using a bigger and more recent web crawl as the document set.
  Mailing list: send a mail message to listproc (at) nist.gov such that the body consists of the line subscribe trec-web <FirstName> <LastName>
  
  
  

• Cross-Language Track
  A track that investigates the ability of retrieval systems to find documents that pertain to a topic regardless of the language in which the document is written.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe xlingual <FirstName> <LastName>
  Cross-Language Track webpage
  While the last cross-language track in TREC was run in TREC 2002, cross-language retrieval tasks are studied in both CLEF (Cross-Language Evaluation Forum), and the NTCIR workshops.
  
  
• Enterprise Track
  The purpose of the enterprise track is to study enterprise search: satisfying a user who is searching the data of an organization to complete some task.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe trec-ent <FirstName> <LastName>
  
  
• Filtering Track
  A task in which the user's information need is stable (and some relevant documents are known) but there is a stream of new documents. For each document, the system must make a binary decision as to whether the document should be retrieved (as opposed to forming a ranked list).
  Mailing list: send a mail message to lyris@list.research.microsoft.com such that the body consists of the line subscribe trecfiltering <FirstName> <LastName>
  The filtering track was last run in TREC 2002.
  
  
• Genomics Track
  The purpose of the track was to study retrieval tasks in a specific domain, where the domain of interest is genomics data (broadly construed to include not just gene sequences but also supporting documentation such as research papers, lab reports, etc.). The Genomics track last ran in TREC 2007.
  Genomics Track webpage
  
  
• HARD Track
  The goal of HARD is to achieve High Accuracy Retrieval from Documents by leveraging additional information about the searcher and/or the search context, through techniques such as passage retrieval and using very targeted interaction with the searcher. The hard track last ran in TREC 2005.
  HARD Track webpage includes details regarding how to join the HARD track mailing list.
  
  
• Interactive Track
  A track studying user interaction with text retrieval systems. Participating groups develop a consensus experimental protocol and carry out studies with real users using a common collection and set of user queries. The interactive track last ran as an adjunct to the Web Track in TREC 2003.
  Mailing list: Contact Ross.Wilkinson@csiro.au to be added to the list.
  Interactive Track webpage
  
  
• Novelty Track
  A track to investigate systems' abilities to locate new (i.e., non-redundant) information. This track last ran in TREC 2004.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe trec-novelty
  <FirstName> <LastName>
  
  
• Question Answering Track
  A track designed to take a step closer to information retrieval rather than document retrieval. The QA track last ran in 2007.
  Mailing list: Send a mail message to listproc@nist.gov such that the body consists of the line
  subscribe trec-qa <FirstName> <LastName>
  QA Track webpage
  
  
• Robust Retrieval Track
  A track that includes a traditional ad hoc retrieval task task, but with the focus on individual topic effectiveness rather than average effectiveness. This track was discontinued after TREC 2005, but the "geometric MAP" measure that resulted from the track is included in trec_eval (.gz) and will be used in other ad hoc tasks.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe robust <FirstName> <LastName>
  
  
• SPAM Track
  The goal of the SPAM track is to provide a standard evaluation of current and proposed spam filtering approaches, thereby laying the foundation for the evaluation of more general email filtering and retrieval tasks. The SPAM track last ran in TREC 2007.
  Mailing list: Send a mail message to listproc@nist.gov such that the body consists of the line subscribe trecspam <FirstName> <LastName>
  
  
• Terabyte Track
  The purpose of the terabyte track is to investigate whether/how the IR community can scale traditional IR test-collection-based evaluation to significantly larger document collections than those currently used in TREC. This track last ran in 2006.
  Mailing list: send a mail message to listproc@nist.gov such that the body consists of the line subscribe trec-tb <FirstName> <LastName>
  Terabyte Track webpage
  
  
• Video Track
  TREC 2001 and 2002 contained a video track devoted to research in automatic segmentation, indexing, and content-based retrieval of digital video. Beginning in 2003, the track became an independent evaluation (TRECVID).
  TRECVID homepage
  
  
• Web Track
  A track featuring search tasks on a document set that is a snapshot of the World Wide Web. This Web track last ran in TREC 2004.
  Mailing list: Contact Nick.Craswell@cmis.csiro.au to be added to the list.
  Web Track webpage