This set of files contains answer patterns for
the TREC 2004 (TREC 13) QA track. The patterns
are given separately for the factoid questions,
the list questions, and the 'other' questions.
The files give both the patterns and the documents
in which assessors scored an answer as 'exact'
(for factoid and list questions) or responsive
to a nugget (for 'other' questions).

Factoid Patterns:
-----------------
Two files are provided: 
	trec13factpats.txt
	trec13factdocs.txt

The actual patterns are provided as Perl patterns
in trec13factpats.txt with lines of the form

	qid pattern

The documents in which these patterns are found
are given in trec13factdocs.txt with lines of 
the form

	qid pattern (docid)*

with zero or more documents reflecting actual
submissions.

Note that there may be more than one line with
the same qid, where the pattern was conceptually distinct.

List patterns:
--------------
Two files are provided: 
	trec13listpats.txt
	trec13listdocs.txt

The actual patterns are provided as Perl patterns
in trec13listpats.txt with lines of the form

	qid pattern

Note that the patterns began with the list
answers as provided by NIST in list_answers.txt.
However, the patterns were modified to reflect
the actual answers scored as correct. Generally,
the new patterns were added as 'or' options in 
the Perl pattern and kept in the same pattern
so that variants of the same basic answer all
are found in the same pattern. The original
NIST answer is retained for reference purposes.

The documents in which these patterns are found
are given in trec13listdocs.txt with lines of 
the form

	qid pattern (docid)*

with zero or more documents reflecting actual
submissions.

Note that, in this file, pattern and the set
of docid's are separated by a tab.

'Other' patterns:
-----------------
One file is provided:
	trec13otherdocs.txt

This file does not contain actual patterns.
It contains the answer strings that were judged
responsive to the NIST list of vital and acceptable
nuggets for each 'other' question. This file was
created beginning with the NIST supplied nuggets
in other_answers.txt. Under each nugget, a list of
answers judged to contain the nugget is given.
These lines are of the form

	qid nugnum docid string

where qid and nugnum may have a leading zero
(used for sorting purposes).  In creating this
file, duplicate answer strings were removed.
However, since the actual submissions may vary
slightly, virtual duplicates may still remain.
Creation of actual Perl patterns for each nugget
has not been completed and may require a little
work.

Please address any questions pertaining to these
files to Ken Litkowski (ken@clres.com) and not to 
NIST.