The Text Retrieval Conference (TREC) workshop series encourages
research in information retrieval and related applications by
providing a large test collection, uniform scoring procedures,
and a forum for organizations interested in comparing their
results. Now in its 21st year, the conference has become
the major experimental effort in the field. Participants in
the previous TREC conferences have examined a wide variety
of retrieval techniques and retrieval environments,
including cross-language retrieval, retrieval of web documents,
multimedia retrieval, and question answering. Details about TREC
can be found at the TREC web site, http://trec.nist.gov.
You are invited to participate in TREC 2012. TREC 2012 will
consist of a set of tasks known as "tracks". Each track focuses
on a particular subproblem or variant of the retrieval task as
described below. Organizations may choose to participate in any or
all of the tracks. Training and test materials are available from
NIST for some tracks; other tracks will use special collections that
are available from other organizations for a fee.
Dissemination of TREC work and results other than in the (publicly
available) conference proceedings is welcomed, but the conditions of
participation specifically preclude any advertising claims based
on TREC results. All retrieval results submitted to NIST are
published in the Proceedings and are archived on the TREC web site.
The workshop in November is open only to participating groups that
submit retrieval results for at least one track and to selected
government invitees.
Schedule:
Schedule:
As soon as possible -- submit your application to participate in
TREC 2012 as described below.
Submitting an application will add you to the active participants'
mailing list. On Feb 23, NIST will announce a new password
for the "active participants" portion of the TREC web site.
Beginning March 1
Document disks used in some existing TREC collections distributed
to participants who have returned the required forms.
Please note that no disks will be shipped before March 1.
July--August
Results submission deadline for most tracks
Specific deadlines for each track will be included in
the track guidelines, which will be finalized in the spring.
September 30 (estimated)
relevance judgments and individual
evaluation scores due back to participants.
Nov 6-9
TREC 2012 conference at NIST in Gaithersburg, Md. USA
Task Description
Below is a brief summary of the tasks. Complete descriptions of
tasks performed in previous years are included in the Overview
papers in each of the TREC proceedings (in the Publications section
of the web site).
The exact definition of the tasks to be performed in each track for
TREC 2012 is still being formulated. Track discussion takes place
on the track mailing list. To be added to a track mailing list,
follow the instructions for contacting that mailing list as
given below. For questions about the track, send mail to the
track coordinator (or post the question to the track mailing list
once you join).
TREC 2012 will contain eight tracks. The Crowdsourcing,
Legal, Medical Records, Microblog, Session, and Web tracks will
continue. TREC 2012 will add tracks on Contextual Suggestion
and Knowledge Base Acceleration.
Contextual Suggestion Track
The Contextual Suggestion track investigates search techniques for
complex information needs that are highly dependent on context
and user interests.
Track coordinators:
Charles L A Clarke, claclark (at) gmail.com
Tatiana Gossen, tatiana.gossen (at) ovgu.de
Jaap Kamps, kamps (at) uva.nl
Paul Thomas, paul.thomas (at) csiro.au
Track Web Page:
http://sites.google.com/site/treccontext/
Mailing list:
Send a mail message to listproc (at) nist.gov
such that the body consists of the line
subscribe trec-context <FirstName> <LastName>
Crowdsourcing Track
The Crowdsourcing track investigates emerging crowd-based
methods for search evaluation and/or developing hybrid automation+crowd
search systems.
Track coordinators:
Gabriella Kazai, v-gabkaz (at) microsoft.com
Matt Lease, ml (at) ischool.utexas.edu
Track Web Page:
https://sites.google.com/site/treccrowd2011
Mailing list:
http://groups.google.com/group/trec-crowd
Knowledge Base Acceleration Track
This track looks to develop techniques to dramatically improve
the efficiency of (human) knowledge base curators by having the system
suggest modifications/extensions to the KB based on its monitoring of
the data streams.
Track coordinators:
John R. Frank, jrf (at) mit.edu
Ian Soboroff, ian.soboroff (at) nist.gov
Track Web Page/Mailing List:
http://groups.google.com/group/trec-kba
Legal Track
(the Legal Track will be unable to run in 2012)
The goal of the legal track is to develop search technology
that meets the needs of lawyers to engage in effective discovery
in digital document collections.
Track coordinators:
Gordon V. Cormack, gvcormac (at) uwaterloo.ca
Maura R. Grossman, MRGrossman (at) wlrk.com
Bruce Hedin, bhedin (at) h5.com
Douglas W. Oard, oard (at) umd.edu
Track Web Page:
http://trec-legal.umiacs.umd.edu
Mailing list:
Contact oard (at) umd.edu to be added to the list.
Medical Records Track
The goal of the Medical Records track is to foster research on
providing content-based access to the free-text fields of
electronic medical records.
Track coordinators:
Ellen Voorhees, ellen.voorhees (at) nist.gov
Mailing list:
http://groups.google.com/group/trec-med
Microblog Track
The Microblog track examines search tasks and evaluation
methodologies for information seeking behaviors in
microblogging environments.
Track coordinators:
Iadh Ounis, Jimmy Lin, Craig Macdonald, and Ian Soboroff
trec-microblog-organisers (at) googlegroups.com
Track Web Page:
http://sites.google.com/site/trecmicroblogtrack/
Mailing list:
http://groups.google.com/group/trec-microblog
Session Track
The Session track aims to provide the necessary resources in the
form of test collections to simulate user interaction and help evaluate
the utility of an IR system over a sequence of queries and user
interactions, rather than for a single "one-shot" query.
Track coordinators:
Ben Carterette, carteret (at) cis.udel.edu
Paul Clough, p.d.clough (at) sheffield.ac.uk
Evangelos Kanoulas, e.kanoulas (at) sheffield.ac.uk
Mark Sanderson, mark.sanderson (at) rmit.edu.au
Track Web Page:
http://ir.cis.udel.edu/sessions
Mailing list:
Use the link given on the track web page to join
the email list.
Web Track
The Web track explores Web-specific retrieval tasks,
including diversity and efficiency tasks, over collections of
up to one billion Web pages.
Track coordinators:
Charles Clarke, claclark (at) plg.uwaterloo.ca
Nick Craswell, nickcr (at) microsoft.com
Track Web Page:
http://plg.uwaterloo.ca/~trecweb
Mailing list:
Send a mail message to listproc (at) nist.gov
such that the body consists of the line
subscribe trec-web <FirstName> <LastName>
Conference Format
The conference itself will be used as a forum both for presentation
of results (including failure analyses and system comparisons),
and for more lengthy system presentations describing retrieval
techniques used, experiments run using the data, and other issues
of interest to researchers in information retrieval.
All groups will be invited to present their results in a joint
poster session. Some groups may also be selected to present
during plenary talk sessions.
Application Details
Organizations wishing to participate in TREC 2012 should respond
to this call for participation by submitting an application.
Participants in previous TRECs who wish to participate
in TREC 2012 must submit a new application.
To apply, follow the instructions at
http://ir.nist.gov/trecsubmit.open/application.html
to submit an online application. The application system
will send an acknowledgement to the email address
supplied in the form once it has processed the form.
Any questions about conference participation should be sent
to the general TREC email address, trec (at) nist.gov.
|
TREC home
