Mobile Camera-Based Systems for Low-Resource Environments

Abstract

The suitability of mobile devices for data collection and decision support in developing countries has been well established. It is now relatively common for field workers to carry devices that help them decide what questions to ask and how to record the resulting data. Many existing data collection tools also allow users to gather rich data from built-in sensors, such as taking photos using the device's camera. Typically, this data is simply stored on the device or uploaded to a server for analysis. However, there are many potential benefits to be gained by analyzing data collected from sensors immediately on the device, such as lowering the cost and bandwidth required for data transmission, enabling immediate analysis in the absence of reliable connectivity, and delivering the results of analysis more quickly than waiting for responses from an external server or off-site human expert. This dissertation describes the creation of systems that run on commercially available mobile devices and that use camera-based input in conjunction with computer vision and machine-learning to improve data collection and disease diagnosis in remote areas. In particular, we: - Show that commercially available mobile devices are capable of capturing high-quality images and videos that can be processed using computer vision and machine-learning techniques running locally on the device. - Identify and overcome the technical challenges associated with designing and implementing algorithms to interpret camera-based input on the device and quantify the technical performance of these algorithms. - Develop an approach to building camera-based systems that allows users to specify how the system should interpret images without needing to recompile the software. - Demonstrate the viability of our approach by applying our methods to two different problem domains: automatically digitizing data from paper forms, and automatically interpreting diagnostic tests. - Prove that our systems can be effectively integrated into existing information ecosystems in low-resource settings and demonstrate that they are usable and appropriate under the constraints experienced at all levels of the information hierarchy. Taken together, these contributions demonstrate that mobile, camera-based systems could alleviate some of the burdens faced by field workers in low-resource settings. Moreover, in addition to identifying and overcoming the technical challenges associated with building systems for low-resource environments, we have also developed a deep understanding of the human challenges - cultural, linguistic, and social - that impact the research process. Through understanding and tackling these challenges, this dissertation contributes a new approach for designing and building technologies for underserved communities and provides evidence for how this approach can be used to strengthen information and healthcare systems in developing countries.