Cognitive Mechanisms for Reasoning with Uncertainty Visualizations

Abstract

The joint proliferation of data-driven interfaces in public life and data science in organizations makes reasoning with uncertainty in data visualizations critically important. Lay people and data analysts alike make visual judgments about data almost daily---whether relying on a deluge of Covid-19 visualizations to manage risks to personal and public health, or using exploratory data analysis to drive business decisions. In order to design data visualization software that supports statistically rigorous judgments in these contexts, the visualization community must understand how people reason with uncertainty visualizations. My dissertation addresses cognitive mechanisms that chart users rely on when reasoning with uncertainty: (1) automatic perceptual processing, through which the visual system makes intuitive inferences; (2) heuristic strategies, used to interpret visualizations and make consequential decisions; and (3) model-based thinking, whereby analysts compare observed patterns in data with counterfactual predictions from models (either mental or realized in software) that might explain the data. As a capstone to my thesis, I present Exploratory Visual Modeling (EVM), a prototype visual data analysis tool that deploys these cognitive mechanisms to support more rigorous exploratory data analysis. The tool enables analysts to express their provisional mental models of data generating process as formal statistical models and to check predictions from these models against observed patterns in data. I present insights from the design process of EVM, as well as considerations for evaluating the design hypothesis that the model checks enabled by EVM facilitate improvements in generative thinking during exploratory data analysis. EVM deploys automatic and heuristic cognitive mechanisms in service of model-based thinking, providing a proof-of-concept for new ways of designing visualization software.