The Effect of Visual Design and Information Content on Readers’ Assessments of API Reference Topics

Abstract

Software developers must learn and use an increasing number of application-programming interfaces (APIs) to create applications and web sites. To apply these APIs in the increasingly short development time that modern markets require, software developers must learn to use APIs quickly. In the process of learning and using APIs, software developers must find and evaluate many API documentation topics, which can vary greatly in their visual design and the information they contain. This study applied a task-based, experimental methodology to measure the effects that variations in the visual design and information concepts used in API reference topics have on software developers’ speed and accuracy when they assess the topics’ relevance, and their perceptions of the topics. In an Internet-based, remote study, participants performed four information-seeking tasks in which they decided whether an API reference topic was relevant to a question presented in a typical programming scenario. The study analyzed 698 individual information-seeking tasks from 201 software developers who lived in 30 different countries and who were proficient in English. Variations in the visual design elements of API reference topics did not significantly affect the time required to assess the topics’ relevance to the information-seeking task yet the variations significantly influenced participants’ assessments of the topics’ credibility and professional appearance. Variations in the information concepts presented in the API reference topics, on the other hand, significantly influenced both the time participants took to evaluate the topics’ relevance and the participants’ assessments of the topics’ credibility and professional appearance. This study contributes critically needed empirical data in an under-studied area concerning how document design elements influence readers’ performance and perception of API reference topics. This study also updates best-practice recommendations for practitioners who write API reference topics to help them prioritize their documentation efforts. Finally, the study provides information about tools and methods that could provide guidance for testing and improving API documentation and other types of documents and in other contexts.