Human-Centered Interactive Information Seeking

Abstract

The rapidly increasing volume of accessible information in the digital age underscores the need for developing automated models to facilitate our daily information-seeking tasks. Moreover, as humans often have exploratory information needs and do not always come up with very descriptive initial queries, it is important to build an AI agent that can converse and collaborate with human users to find the information they are interested in. Upon receiving and interpreting a user's request in a conversation, the AI agent should compose a response grounded in relevant information from a knowledge source. In this thesis, we introduce our proposed solutions to three pivotal challenges in this interactive process. First, identifying relevant knowledge to be used in conversational systems that are grounded in long documents is critical to effective response generation. We introduce a knowledge identification model DIALKI, that leverages the document structure to provide dialogue-contextualized passage encodings and better locate knowledge relevant to the conversation. An auxiliary loss captures the history of dialogue-document connections. We demonstrate the effectiveness of DIALKI on two document-grounded conversational datasets and provide analyses showing generalization to unseen documents and long dialogue contexts. When the knowledge source contains millions or more documents or passages, passage retrieval is performed before response generation. As it is often not practical to re-train well-established retrievers like a search engine to handle conversational queries specifically, we develop a query rewriting model CONQRR that rewrites a question that depends on dialogue context in the context into a standalone question. It is trained with a novel reward function to directly optimize towards retrieval using reinforcement learning and can be adapted to any off-the-shelf retriever. CONQRR achieves state-of-the-art results on a recent open-domain conversational question answering dataset containing conversations from three different sources, and it is effective for two different off-the-shelf retrievers. Our extensive analysis also shows the robustness of CONQRR to out-of-domain dialogues as well as to a scenario where no query rewriting supervision is available. The second challenge comes from the need for AI agents to collaborate with users during the information-seeking process. To facilitate this line of research, we presents INSCIT, a dataset for Information-Seeking Conversations with mixed-initiative Interactions. It contains 805 human-human conversations where the agent searches over Wikipedia and either directly answers, asks for clarification, or provides relevant information to address user queries. Along with the data, we propose two subtasks, evidence passage identification and response generation, as well as a human evaluation protocol to assess model performance. We develop two strong baselines. Both of them significantly underperform humans, suggesting ample room for improvement in future studies. Finally, to teach an AI agent to generate responses that can provide the maximized utility to the user, we propose to use fine-grained human feedback (e.g., which sentence is false, which sub-sentence is irrelevant) as an explicit training signal. We introduce fine-grained RLHF, a framework that enables training and learning from reward functions that are learnt from such fine-grained human feedback. Our experiments illustrate how learning with such reward functions leads to improved performance in response generation for information queries, supported by both automatic and human evaluation. Additionally, we show that LM behaviors can be customized using different combinations of fine-grained reward models.