Cultural Adaptation and Evaluation of LLM-Driven Mental Health Conversational Agents

Abstract

Mental health disparities disproportionately affect underserved family caregivers, due in part to the limited availability of culturally responsive interventions. While large language model (LLM)-driven conversational agents hold promise for scalable mental health support, their responses often lack cultural responsiveness, undermining empathy, therapeutic alliance, and engagement among diverse populations. This dissertation develops and evaluates a novel approach for dynamic cultural adaptation of LLM-based mental health agents leveraging context engineering. The process began with stakeholder engagement to identify culturally salient caregiving challenges among Chinese American family caregivers (Aim 1). In collaboration with domain experts, a cultural context database was developed to capture these challenges alongside culturally responsive response examples, enabling real-time retrieval and integration of relevant context during agent interactions. To evaluate this approach, I conducted a contextualized, multiphase evaluation through a series of user studies with Chinese American and Latino American family caregivers. Findings show that the context-engineered system consistently generated responses rated as more culturally responsive and empathic than both a prompt-based adaptation strategy and a baseline non-adapted agent (Aim 2). In a randomized user study with Chinese American caregivers (Aim 3), the culturally adapted agent significantly improved near-term emotional well-being and received higher ratings for cultural competence and therapeutic alliance compared to the non-adapted agent. Notably, participants indicated greater willingness to recommend the adapted agent within their communities. Together, this work contributes: (1) a workflow for adapting LLM-based conversational agents to diverse populations, (2) empirical evidence that cultural adaptation enhances user outcomes in mental health agents, and (3) a human-centered evaluation procedure for assessing cultural responsiveness of AI in mental health contexts. While future research is needed to refine and validate this approach across additional cultural groups and care domains, the findings represent a significant step toward advancing equitable, culturally responsive digital mental health support and reducing persistent disparities in access and quality of care.