Human-AI Mechanisms for Scholarly Knowledge Synthesis

Abstract

The rapid growth in research publication presents significant challenges for researchers trying to stay current with relevant literature. This information overload risks duplicated research efforts, limits innovation, and hinders scholarly growth as researchers struggle to identify gaps and contextualize their work within the existing body of knowledge. While recent advances in AI offer potential solutions to support researchers in engaging with the literature, effectively integrating these techniques into existing research processes requires a richer understanding of how these AI mechanisms can both enhance and threaten researchers' cognitive processes. In this thesis, I examine how new human-AI tools can support researchers in reading, reviewing, and synthesizing research literature more effectively to inform and drive future scientific endeavors. I propose structured mediation as a design framework for human-AI collaboration in scholarly sensemaking that creates intermediate, interpretable representations to enhance researcher capabilities while preserving agency and enabling verification. This framework operates through three core mechanisms: discovery (AI-provided information scent that guides attention and exploration), iteration (collaborative refinement of data gathering and conceptual frameworks), and verification (maintaining clear provenance to source material through structured representations). I present three systems that demonstrate this framework, each exploring interactive, verifiable AI-powered mechanisms that serve as cognitive scaffolds to support researchers' existing literature review processes. First, I explore a new interaction paradigm in recursive, just-in-time expansion of paper abstracts that enables researchers to retrieve clarifying information from full texts of papers during triage with new information presented in-context and attributed to the source text. Then, within the context of a paper's full text, I demonstrate how AI-suggested faceted highlights can aid researchers in more rapidly gathering relevant information by judiciously directing their attention while reading. Finally, intermediate structured representations progressively transform large sets of unstructured research papers into familiar sensemaking schemas, such as tables, hierarchies, and textual summaries, that support steerable and transparent exploration, comparison, and synthesis of information from papers at scale. Through lab and deployment user studies, I demonstrate how these interactive, human-AI systems offer valuable cognitive scaffolds that augment researchers' abilities to navigate across, drill down into, and make sense of dense, complex information within research papers. The result is more effective and efficient literature navigation and synthesis that enhances rather than replaces human judgment in the research process. This work offers guidance for designing mixed-initiative systems that foster effective human-AI collaboration across the review and synthesis of scientific knowledge.