Making Chatbots More Transparent and Applicable to New Demographics

Abstract

Conversational agents, popularly called chatbots, received significant attention in the last few years. The major reason behind the success of these systems is that chatbots use the already familiar conversational interface. However, chatbots are still in their nascent stage: They have a low penetration rate as 84% of the Internet users have not used a chatbot yet. First, we conducted a study with 16 first-time chatbot users interacting with eight chatbots over multiple sessions on the Facebook Messenger platform. Analysis of chat logs and user interviews revealed several major problems with the current chatbots, including (a) mismatch between the chatbot's state of understanding (also called context) and the user's perception of the chatbot's understanding, (b) limitations in natural language understanding technologies leading to dialog failures, and (c) targeting chatbots specifically towards the Internet-savvy technically-advanced users. Second, we focused on these three problems and developed solutions, respectively: (a) Convey: stands for CONtext View, is a window added to the chatbot interface, displaying the conversational context and providing interactions with the context values, which we evaluated with 16 participants; (b) Resilient Chatbot: explores user preferences for eight repair strategies taken from commercially-deployed chatbots (e.g., confirmation, providing options) as well as novel strategies explaining characteristics of the underlying machine learning algorithms, was evaluated with 216 MTurkers; and (c) FarmChat: is a multi-modal multi-lingual conversational agent, to meet the information needs of rural low literate farmers, and evaluated with 34 farmers in Ranchi, India. To summarize, we propose ways to make chatbots more transparent, and extend its applicability to new demographics.