Prediction and Inference on Big Data in Development

Abstract

Recent years have seen an explosion of large-scale data sets pertinent to developing regions. The interest now being paid to country-wide satellite imagery and mobile network data has strong parallels to the proliferation of earlier work being performed on datasets such as ImageNet and the Facebook social network. The hope is that the techniques developed to process and analyze the data in this first iteration of Big Data can be now be turned to datasets from developing regions. Applications in data science for development include increasing business efficiency and competitiveness in these regions, as well as directly improving human development and well-being. This thesis seeks to make Big Data work for applications in the developing world through a comparison of several different projects, including predicting regional wealth and inferring the impact of violence from call data, and determining the quality of a road network from satellite imagery. With this breadth of applications and data types, an integrated approach comprising statistics, economics, and machine learning is vital in data science for development.