Reliability of verbal autopsies and its implications for routine cause of death surveillance

Abstract

<bold>Background<bold> Verbal autopsies (VAs) have been widely used to determine cause of death (COD) for research in developing countries. Understanding the quality of these estimates is essential for research and policy making. Quality of VA surveys can be assessed both in terms of validity and reliability. The former has been extensively researched, but there is not much evidence available for the latter. This study aims to determine if VAs yield consistent results by quantifying the test-retest reliability of verbal autopsies and exploring possible factors associated with reliability in the context of VAs. <bold> Methods <bold> For this study we collected two VAs for 2,113 decedents in Bohol, Philippines; Manila, Philippines; and Andhra Pradesh, India using the Population Health Metrics Research Consortium (PHMRC) Verbal Autopsy Instrument (VAI). COD was predicted using the Tariff Method for VA analysis. Reliability was measured for question responses, COD predictions for individual deaths, and predicted cause-specific mortality fractions (CSMFs). Factors associated with reliability of VA question responses and COD predictions were examined in a regression framework. <bold> Results <bold> We found that although responses to specific questions were often unreliable, there was a much greater degree of reliability for cause of death estimates, particularly at the population-level. Both the interviewer and respondent were found to have significant effects on the reliability of VA questions. We also found the reliability of question responses had a significant effect on the reliability of COD predictions. <bold> Interpretation <bold> Based these results we recommend a greater emphasis be placed on the standardization of VA administration protocols and the training of interviewers. COD estimates derived from VA are essential to informing public health research and policy. Therefore, we must work to ensure that the COD predictions and the survey data underlying them are as reliable as possible.