Grounded Design of Affective Computing Accounting for Social, Emotional, and Sensory Experiences of Autistic Adults

Abstract

We use digital communication technologies to augment or replace face-to-face interactions in our work and daily lives. People’s emotions and affective responses play a significant role in establishing mutual understanding. Technologies such as video calling and chat bots allow people to express themselves in implicit and explicit ways, such as through facial expressions and vocal tone. As digital technologies increasingly mediate emotionally-rich communication, technology is called upon to become emotionally aware and detect, transmit, and respond to emotional cues. This dissertation examines affective computing through the lens of autism. The phenomenon of autism—with its unique expressions of cognitive, sensory, and social styles—offers the research community valuable perspectives for designing inclusive communication and affective computing technologies. According to the Center for Disease Control, around 1 in 54 children in the United States has been diagnosed with autism. This research confronts a crucial concern in technology design: how can we design affective computing that is inclusive of autistic ways of being and does not exacerbate inequities formed by neurotypical social norms and infrastructural barriers. As technologists encode social norms into affective computing, communication technologies, and artificial intelligence, it is imperative that they design experiences that are comfortable for autistic individuals, rather than perpetuate burdens typically placed upon disabled people. This dissertation investigates how to improve the design of affective computing to support face-to-face interpersonal communication between neurodiverse dyads—pairs each composed of an autistic young adult and a non-autistic conversation partner. My research engages autistic young adults to identify problems with communication technologies that they encounter during early adulthood—a time period in which they are taking more ownership of their technology decisions and adopting technological practices of adulthood. Autistic young adults must navigate a world in which the vast majority of people they communicate with daily are neurotypical. The burden is often placed on autistic individuals to modify their behavior to adapt to neurotypical socio-technical norms. During interpersonal interactions, neurodiverse dyads co-construct their emotional and social experiences while communicating across boundaries of neurological differences. Current computer-mediated communication (CMC) and affective technologies do not adequately address neurodivergent individuals due to the lack of scaffolding for non-normative conversations and emotional exchanges. This technology gap is particularly detrimental for autistic individuals, since miscommunication and social tensions contribute to their social isolation, reduced agency, and (more broadly) limited education and employment opportunities. Through Grounded Design research—composed of a context study, participatory design, and technology appropriation—this dissertation engages autistic research participants as co-designers in envisioning affective computing that works for their communication needs. Empirically-based insights informed conceptual contributions of this work, including a social-emotional-sensory design map that highlights the embodied and co-constructed nature of the emotional experiences of autistic adults within the context of physical environments, social relationships, and technology use. The design map serves as a conceptual tool to help designers and researchers recognize rich design sites to improve affective computing for neurodiverse communities. This work also contributes a speculative design concept and prototype of an emotion translator that explored alternative ways to augment a conversation with rich visual imagery to convey emotional states. Through design and appropriation studies, participants conveyed their conceptualization of emotions and often unspoken social norms of conversations. By appropriating a low-fidelity prototype of a chat with “Wizard of Oz” emotional translation functionality, neurodiverse dyads preserved, re-configured, and critiqued the emotion translator. This dissertation contributes empirically based insights, a theoretical framework, and design artifacts that expand scholarly knowledge of how neurodivergent young adults experience social interactions and emotional states. By presenting deeper understanding of neurodivergent experiences and offering design strategies, this works opens up design horizons for more inclusive affective computing and participatory design.