Incompleteness
Study by Zobel [SIGIR-98]:
- Quality of relevance judgments does depend on pool depth and diversity
- TREC ad hoc collections not biased against systems that do not contribute to the pools
- TREC judgments not complete
- additional relevant documents distributed roughly uniformly across systems but highly skewed across topics