Increasing the utilization of administrative data in low- and middle-income countries: a case study for mapping vaccination

Abstract

Childhood vaccination is one of the most cost-effective public health interventions, yet over 30 million children under five in low- and middle-income countries (LMICs) still contract vaccine preventable diseases each year, with more than half a million dying. Reaching the equity goals of Immunization Agenda 2030 requires reliable subnational data to identify under vaccinated children, monitor coverage trends, and target resources. In practice, two data streams dominate this evidence base: household surveys, which are accurate but infrequent as they are expensive, and administrative data routinely collected through health information systems such as the District Health Information Software 2 (DHIS2) and also reported globally through the World Health Organization and United Nations Children’s Fund (WHO/UNICEF) Joint Re porting Form (JRF). Given that surveys are sparsely sampled at lower administrative levels, and recent disruptions in global survey funding have increased reliance on administrative data, concerns about its quality and decision-making utility remain insufficiently characterized. This dissertation investigates the quality and utility of administrative immunization data at multiple scales—nationally within Ethiopia and globally across LMICs—and when possible develops methods to assess and address the gaps that limit its use for equitable program monitoring. In the first aim, we assessed and adjusted for data quality gaps in Ethiopia’s DHIS2 to generate more reliable district-level estimates of full childhood immunization. We analyzed monthly health facility-level counts of fully immunized children reported through Ethiopia’s DHIS2 from September 2019 to August 2025, covering 22,306 vaccine-delivering facilities. We first adjusted numerator-related data quality gaps, including missingness and temporal inconsistencies, using a three-stage Bayesian hierarchical model of reporting, zero occurrence, and positive counts. We then applied a two-stage adjustment procedure combining gravity-based spatial reallocation and percentile matching informed by 2023 survey regional estimates. We compared two denominator sources: Ministry of Health target populations and WorldPop gridded estimates. Data quality gaps were substantial: 15.5% of facility-month observations were missing, and outliers were common. Raw administrative coverage frequently exceeded 100% and showed near-zero correlation with survey-based estimates at the second and third administrative levels. Following adjustment, correlations increased from 0.08 to 0.37 at the third administrative level and from 0.06 to 0.45 at the second, although some of this improvement may reflect the adjustment methodology itself rather than true signal. Adjusted national full immunization coverage was 42.7%, compared with 80.9% from raw administrative data. Geographic inequality was pronounced: 53.7% of districts fell below 50% coverage, 12.5% fell below 25%, coverage ranged from under 5% in Afar to 94.1% in Addis Ababa, and the 80:20 percentile ratio—coverage in the 80th-percentile district relative to the 20th—was 1.9. In this aim we showed that full childhood immunization coverage in Ethiopia remained low after adjustment, suggesting that reliance on raw administrative data without accounting for underlying data biases may substantially overestimate program performance. Many districts had full immunization coverage below 50%, far below both national and international targets, and marked geographic disparities persisted across the country. The second aim examined whether routine DHIS2 data can capture acute disruptions in immunization service delivery during armed conflict, using the Tigray conflict (2020–2022) and its extension into Afar and Amhara as a case study. To estimate percent-per-month changes in immunization rate we fitted uncontrolled interrupted-time-series (uITS) negative-binomial models for the two directly affected regions of Afar and Amhara—excluding Tigray (lacking immunization data for during the conflict). We also implemented controlled ITS (cITS) negative binomial mixed models using alternative regions (Addis Ababa, Sidama, and Harari) as controls. Using conflict event rates as a continuous exposure, we compared models incorporating various exposure lags to measure the association between conflict events and immunization rates. Using cITS, the monthly rate of change in immunization in conflict-affected regions was 5.3% per month lower than in control regions from conflict onset through November 2021. During the humanitarian response period (December 2021–November 2022), the monthly trend in affected regions increased by 10.0% per month relative to controls. Thereafter, the monthly rate of change in affected regions was 8.7% per month lower than control regions. Results from uITS and other models showed consistent changes in immunization rates associated with conflict. When modeling conflict as a continuous exposure, in Amhara, each doubling of conflict event days in the previous month was associated with a 6.2% decline in immunization rates, with effects concentrated within a two month window. In Afar, the immediate impact was strongest, showing a 5.0% decrease in rates for each doubling in conflict exposure during the current month. In Oromia, conflict events from two months prior were most predictive, corresponding to a 2.7% reduction in immunization rates. The Tigray conflict led to significant short-term declines in immunization coverage in regions directly affected by the conflict. However, the prolonged impacts across multiple health aspects remain to be studied. Effective utilization of DHIS2 for assessing impact of external shocks will require improving its data quality and integrating it with survey-based individual-level data. The third aim broadened the analysis from a single country to global JRF subnational administrative data covering 130 LMICs and four core vaccines, Bacillus Calmette–Guérin, first-dose diphtheria–tetanus–pertussis, third-dose diphtheria–tetanus–pertussis, and first-dose measles containing vaccine, over 2016–2024, evaluating data quality across completeness, internal consistency, and validity, and comparing the resulting program- and policy-relevant indicators with matched survey estimates from 51 survey-years across 32 countries. Data quality gaps were common across administrative levels: 26.3% of second-level DTP1–DTP3 dropout was negative (16.6% at the first level; 11.5% nationally), and 30.0% of reported coverage exceeded 100% by 2024 (19.2% at the first level; 13.0% nationally). Upper-middle-income countries generally had better data quality than low- and lower-middle income countries. Across overlapping country survey-years, concordance between administrative and survey-based coverage was weak. Correlations were negative in 38.5% of second-level and 29.3% of first level comparisons, with median absolute differences of 21.1, 10.7, and 9.9 percentage points at the second, first, and national levels, respectively. Administrative estimates exceeded survey values in 78% of second-level and 73.2% of first-level comparisons. In the few countries where it was possible to analyze trends over time, administrative data and survey-derived estimates disagreed more than half the time (55.1%) on whether coverage had improved or worsened in a given second-level unit, with no substantial improvement upon aggregation. Bottom-quintile overlap was only 13%, and agreement on the top quintile by zero-dose burden was 34%. Dropout agreement was also weak, with correlations below 0.5 in most country years and median absolute differences of 14.6 percentage points at the second level (6.2 at the first level), while administrative data systematically underestimated survey-derived dropout rates by 78–89% across all levels. Aggregating administrative data to higher administrative levels, using numerators, and alternative denominators from WorldPop, when available, modestly improved association in some countries. Sensitivity analyses using different survey-based estimates yielded similar conclusions. Subnational administrative immunization data in LMICs exhibit persistent, uneven quality gaps and frequently leading to different programmatic conclusions than survey-based estimates regarding coverage, equity, and zero-dose identification. Addressing these challenges requires country specific investments in routine data quality assessment and further systematic triangulation with external data sources. As Immunization Agenda 2030 (IA2030) positions country-owned data systems as central to equitable immunization, strengthening administrative data quality is a programmatic necessity to ensure that the evidence guiding resource allocation and equity monitoring does not itself amplify the misclassification it seeks to resolve. Across the three aims, the main theme that emerge is the quality of administrative immunization data is often substantially weaker than its prominent role in monitoring would suggest, with errors concentrated in numerators (missingness, outliers, reporting fluctuations) and denominators (outdated census data, mismatched target populations). Therefore, these findings argue for caution in the universal application of subnational administrative data for programmatic and equity decision-making in LMICs and in favor of context-specific data quality assessment, routine triangulation with survey and other external benchmarks, and continued investment in the underlying reporting infrastructure.