Supporting Societal Impact Ideation through LLM-Powered System Design

Abstract

Computing technologies increasingly shape society, yet researchers and practitioners often struggle to anticipate their societal consequences before systems are developed. This dissertation investigates how to support societal impact ideation early in the research and development process through interactive system design. In the first half of this dissertation, I conducted empirical studies with computer science researchers from diverse subfields, showed that practitioners view this work as important yet rarely practice it, and characterized the barriers that explain this gap. I found that researchers lacked exposure to concrete examples of past harms and lacked structure and access to viewpoints beyond their own. These findings motivate the need for individual tooling that can provide relevant examples, surface diverse perspectives, and create structure for reflection. Building on insights from the findings, I developed and evaluated three LLM-powered systems and derived three corresponding design guidelines for supporting societal impact ideation: grounding, structuring, and co-reasoning. First, BLIP grounds reflection in real-world historical cases of technology consequences, allowing practitioners to draw on concrete precedents. Second, PeaceBot structures exploration across diverse stakeholder perspectives by combining guided prompts with simulated stakeholders and an explicit idea curation process. To support the reasoning that follows ideation, Hippo introduces Interactive Reasoning, an interaction design that makes an LLM's intermediate reasoning steps visible and editable, so users can inspect, revise, and steer assumptions before accepting a final output. Together, this work frames societal impact ideation as an interaction design problem that current tools fail to adequately scaffold. Practitioners need concrete material to think with, structures for broad exploration, and opportunities to actively shape model-supported reasoning. The three systems demonstrate how LLM-powered systems can be (re-)designed to support societal impact ideation. I discuss the implications of my work on societal impact ideation, its associated challenges, and the design considerations that emerge when balancing LLM automation with the human agency required for reflective work.