Familiarization to Speech in Noise: The Roles of Phonological Processing and Audiovisual Integration

Abstract

Phonological processing, or the ability to recognize and manipulate phonemes, is a key aspect of language comprehension. Accurate speech recognition relies on the listener's ability to rapidly decode phonemic information. It is known that audiovisual (AV) integration can facilitate the speech recognition process, where simultaneously presented visual cues of a speaker's articulators can calibrate perception of the acoustic characteristics of incoming speech signals. In typically developing listeners, speech recognition is more robust with AV stimulus integration compared to unimodal auditory presentation. However, it remains unclear whether AV integration similarly benefits speech recognition in listeners with poor phonological processing ability. The present study uses a high-level linguistic task to examine whether a listener's recognition of unfamiliar speech in noise is moderated by their phonological processing ability, given a familiarization passage with unimodal auditory or multimodal AV stimulus presentation. Results show that there was no benefit of passive AV exposure to the speaker for sentence recognition. Moreover, while phonological processing ability supports overall recognition of speech in noise, it did not moderate the amount of gain received by AV integration. These findings indicate that as a listener-specific factor, phonological processing does not impact the effect of AV integration for the perceptual learning of unfamiliar speech.