Evaluation by Patterns

• Problems associated with patterns:

  • don�t differentiate between documents
  • don�t penalize �answer stuffing�
  • errors correlated with system functionality

• Useful if limitations understood

  • system rankings produced from lenient- assessor and pattern evaluations highly correlated
    • Kendall t of .944 (24 swaps) for 250-byte
    • Kendall t of .894 (28 swaps) for 50-byte

Previous slide

Next slide

Back to first slide

View graphic version