QA Main Task
Return a ranked list of 5 response pairs for each of 500 questions
- questions drawn from MSNSearch and AskJeeves logs
- no guarantee that question has answer in collection, so a response could be `NIL’
- also returned a single “final answer”
- either a rank or the string `UNSURE’
Evaluated using mean reciprocal rank
- strict scoring: unsupported counted as wrong