Using Visual Analytics to Support Artificial Intelligence Development
MetadataShow full item record
In the artificial intelligence (AI) era, complex AI is becoming more common in daily life. Various reports indicate that AI plays an increasingly influential role in human society (e.g., [Russell et al. 2015, Stone et al. 2016]). However, the development of a sophisticated AI system is a challenging process. AI researchers must experiment with different AI system architectures and evaluate their performance on the target tasks. As components in the system interact with each other, it can be difficult to make sense of evaluations and find insight to improve the system. Traditional metrics for system performance (e.g., task accuracy) often fail to provide useful and actionable insights for AI developers. The burden for AI researchers is to analyze evaluations to understand how changes impact system behavior, and then find ways to enhance the system. Visualization is a powerful tool to present information in a way that is easier for humans to absorb than text [Little 2015]. Given that AI development involves a vast amount of information, visualization can facilitate insight discovery for AI developers. Among all possible visualization approaches, visual analytics is that which specifically focuses on using visualization to support analytical tasks and is a powerful method for insight discovery. However, designing visual analytics tools is not a trivial task. In particular, since AI development is highly specialized and requires a significant amount of experience, designing visual analytics tools to satisfy AI developers’ needs is challenging. Human-centered design is a powerful technique to address such needs. Therefore, in this thesis, we focus on using a human-centered design approach to study, design, and engineer visual analytics tools to support AI system development. The thesis includes the design of two visualization tools: QSAnglyzer, a visual analytics tool for evaluation analysis; and AnchorViz, an interactive visualization for discovering errors in interactive machine learning classifiers. In addition, based on an interview study, we propose a framework to describe the current practice of AI development workflows, highlighting issues and suggesting design implications for designers and researchers. By using a human-centered design approach, this thesis aims to contribute to the fields of human-computer interaction (HCI), visualization (VIS), and visual analytics (VA) about AI developers and AI development processes, as well as how to design visual analytics for this domain.Our ultimate goal is to pave a road for creating better tools for AI developers, lowering the barriers for AI development, and making AI more accessible to a wider range of people to build and use.