The use of crowdsourcing and the role of game mechanics in identifying erroneous disease burden estimates

Abstract

Objectives: This project evaluates the feasibility of using crowdsourcing techniques for error detection in public health research. Secondarily, it estimates the effect of gamification on the accuracy and volume of disease burden classifications. Methods: The Global Burden of Disease 2010 estimates served as a database for this project. Algorithms were used to identify potentially erroneous estimates. Two user interfaces (one gamified) were designed to collect non-expert disease burden classifications, which were compared against a GBD expert. Results: The 43 participants classified 1,114 health trends using the web interface. Of these, 86% of responses matched the classification of a GBD expert, yielding a sensitivity of .71, and a specificity of .89. The presence of a gamified environment increased usage by 70%. Conclusions: Non-experts were able to accurately classify disease trends. The use of crowdsourcing may be applicable to similarly large databases, and gamification can drive increased use.