ASR Metrics

• Traditional ASR Metric:

  • Word Error Rate (WER) and Mean Story Word Error Rate (SWER) using SCLITE and LDC ref transcripts
  • WER = word insertions + word deletions + word substitutions
    
    total words in reference
    
  • LDC created 2 Hub-4 compliant 10-hour subsets for ASR scoring and analyses (LDC-SDR-99 and LDC-SDR-2000)
    • Note that there is a 10.3% WER in the collection human (closed caption) transcripts

Note: SDR recognition is not directly comparable to Hub-4 benchmarks due to transcript quality, test set selection method, and word mapping method used in scoring

Previous slide

Next slide

Back to first slide

View graphic version