Main Web Relevance Judgments
Up to 4 runs judged per group
- 2 content-only runs and 2 content-link runs
Three relevance levels used
- highly relevant, relevant, not relevant
Assessors also selected best page in pool
- definition of best completely up to assessor
- abstentions allowed, as was multiple bests
Official scoring used standard binary judgments