Information and communication technology for development (ICT4D) evaluation: Examining its foci, gaps, challenges, and associations with other areas to strengthen the field’s conceptual underpinnings

Abstract

The effectiveness of information and communication technology for development (ICT4D) projects has been deeply questionable over the past decades, despite the much attention paid to ICTs among academics and practitioners as a possible catalyst in the betterment of people’s lives in the Global South. Significant failures during the early phases of ICT4D era led to increased attention to evaluation of the projects—the lessons from which can contribute toward increased potential of successful and sustainable ICT4D initiatives. However, ICT4D evaluation is yet to be seen as a field with a clear global consensus on what it constitutes. In this setting, the study aims to identify the major foci, gaps, and challenges of ICT4D evaluation and explore ways to strengthen its conceptual elements and resolve prevailing issues. To address the above research problem, content analysis of journal articles and aid/development agency reports on ICT4D evaluation and interviews of ICT4D researchers in academic and practitioner spheres were conducted. Moreover, to identify and strengthen the field’s conceptual elements, the study refers to the three areas of evaluation with a long history of rich discourse and/or strong foundation of models, theories, and approaches—program evaluation, information systems/information technology (IS/IT) evaluation, and aid/development evaluation. The study finds that: there have been changes in foci in ICT4D evaluation in terms of development notions, research or evaluation approaches, and methods; challenges exist in terms of structural, stakeholder, and methodological issues; there have been associations between ICT4D evaluation and the three evaluation fields but with limitations; and that more discussion is needed on areas such as sustainability, participatory evaluation design, and ethical considerations of researchers. Also, the study argues for the significance of socio-technical lens and development notion–impact connection in addressing the current challenges, and that domain-oriented consolidation of knowledge in ICT4D evaluation among the researchers may help facilitate engagement with decision makers. The study can serve as a ground for facilitating discussions on how the field can be improved and how it can accommodate various perspectives and interests of the ICT4D community.