Automatic sentence structure annotation for spoken language processing
Hillard, Dustin Lundring
MetadataShow full item record
Increasing amounts of easily available electronic data are precipitating a need for automatic processing that can aid humans in digesting large amounts of data. Speech and video are becoming an increasingly significant portion of on-line information, from news and television broadcasts, to oral histories, on-line lectures, or user generated content. Automatic processing of audio and video sources requires automatic speech recognition (ASR) in order to provide transcripts. Typical ASR generates only words, without punctuation, capitalization, or further structure. Many techniques available from natural language processing therefore suffer when applied to speech recognition output, because they assume the presence of reliable punctuation and structure. In addition, errors from automatic transcription also degrade the performance of downstream processing such as machine translation, name detection, or information retrieval. We develop approaches for automatically annotating structure in speech, including sentence and sub-sentence segmentation, and then turn towards optimizing ASR and annotation for downstream applications.The impact of annotation is explored at the sentence and sub-sentence level. We describe our general approach for predicting sentence segmentation and dealing with uncertainty in ASR. A new ASR system combination approach is described that improves ASR more than any previously proposed methods. The impact of automatic segmentation in machine translation is also evaluated, and we find that optimizing segmentation directly for translation improves translation quality, performing as well (or better than) using reference segmentation. Turning to sub-sentence annotation, we describe approaches for supervised comma detection and unsupervised learning of prosodic structure. The utility of automatic commas is then assessed in the context of information extraction and machine translation. Including commas in information extraction tasks significantly improves performance, especially when punctuation precision and recall are optimized directly for entity and relation extraction. We then also propose approaches for improving translation reordering models with cues from commas and sentence structure.
- Electrical engineering