Human-Centered Sound Recognition Tools for Deaf and Hard of Hearing People

Abstract

Sound carries rich information about our surroundings; however, this information can be inaccessible to people who are Deaf, deaf, or hard of hearing (DHH). Automatic sound recognition features are now available on smartphones and other devices, but current implementations offer limited personalization options, hindering their ability to accommodate DHH users' diverse interests and varied contexts of use. In this dissertation, I present a series of iterative studies that explore the design of human-centered sound recognition tools to enable DHH users to tailor sound information to their individual needs. I examine user experiences across different stages of a machine learning workflow for personalization, including: problem framing for contextual sound feedback, capturing and curating personal audio data, and interactive training and evaluation of custom sound recognition models. Together, this work provides a comprehensive, empirical investigation into the challenges and opportunities with developing human-centered sound recognition tools for DHH users. I close with recommendations for interface design and opportunities for future research in this space.