Test Conditions
Primary Conditions (may use non-lexical side info, but must run contrast below):
- R1SU: Reference Retrieval, short topics, using human-generated "perfect" transcripts without known story boundaries
- R1TU: Reference Retrieval, terse topics, using human-generated "perfect"
- B1SU: Baseline Retrieval, short topics, using provided recognizer transcripts without known story boundaries
- B1TU: Baseline Retrieval, terse topics, using provided recognizer transcripts without known story boundaries
- S1SU: Speech Retrieval, short topics, using own recognizer without known story boundaries
- S1TU: Baseline Retrieval,terse topics, using provided recognizer transcripts without known story boundaries
Optional Cross-Recognizer Condition (may use non-lexical side info, but must run contrast below):
- CRSU-<SYS_NAME>: Cross-Recognizer Retrieval, short topics, using other participants' recognizer transcripts without known story boundaries
- CRTU-<SYS_NAME>: Cross-Recognizer Retrieval, terse topics, using other participants' recognizer transcripts without known story boundaries
Conditional No Non-Lexical Information Condition (required contrast if non-lexical information is used in other conditions):
- R1SUN: Reference Retrieval, short topics, using human-generated "perfect" transcripts without known story boundaries, no non-lexical info
- R1TUN: Reference Retrieval, terse topics, using human-generated "perfect" transcripts without known story boundaries, no non-lexical info
- B1SUN: Baseline Retrieval, short topics, using provided recognizer transcripts without known story boundaries, no non-lexical info
- B1TUN: Baseline Retrieval, terse topics, using provided recognizer transcripts without known story boundaries, no non-lexical info
- S1SUN: Speech Retrieval, short topics, using own recognizer without known story boundaries, no non-lexical info
- S1TUN: Speech Retrieval, terse topics, using own recognizer without known story boundaries, no non-lexical info
- S2SUN: Speech Retrieval, short topics, using own second recognizer without known story boundaries, no non-lexical info
- S2TUN: Speech Retrieval, terse topics, using own second recognizer without known story boundaries, no non-lexical info
Optional Known Story Boundaries Conditions:
- R1SK: Reference Retrieval, short topics, using human-generated "perfect" transcripts with known story boundaries
- R1TK: Reference Retrieval, terse topics, using human-generated "perfect" transcripts with known story boundaries
- B1SK: Baseline Retrieval, short topics, using provided recognizer transcripts with known story boundaries
- B1TK: Baseline Retrieval, terse topics, using provided recognizer transcripts with known story boundaries
- S1SK: Speech Retrieval, short topics, using own recognizer with known story boundaries
- S1TK: Speech Retrieval, terse topics, using own recognizer with known story boundaries
Recognition Language Models:
- FLM: Fixed language model/vocabulary predating test epoch
- RLM: Rolling language model/vocabulary using daily newswire adaptation