Strengthening quality management systems of clinical laboratories in Cambodia and the role of quality mentors: a quantitative evaluation of program outputs and outcomes

Abstract

Programs to improve laboratory quality management systems (LQMS) in resource-limited countries have been shown to be an effective means to meet international accreditation standards for improved health outcomes. In the context of Cambodia, where most clinical laboratories have not yet met these standards, much can be learned from the evaluation of current LQMS improvement programs and their effectiveness at achieving measurable change. Toward this end, this study describes and evaluates the methods, outputs, and outcomes of the International Training and Education Center for Health’s Laboratory Health Systems Strengthening Program in 12 clinical laboratories between January 2018 to April 2019. The program goal is to improve laboratory operations in Cambodia for enhanced disease detection, surveillance, and biosecurity through improved quality assurance and management practices. In participating laboratories, the program used a combination of methods, including formal LQMS training, on-site technical consults, and remote mentoring by trained quality mentors through video conference technologies. For evaluation of this program within the logical framework of program activities in participating laboratories, activity outputs were quantified from the activity reports of program trainers and mentors; direct program outcomes were measured by pre- and post-implementation audits of LQMS conformity to international standards; and changes in laboratory quality as indirect outcomes were assessed through a set of common indicators of laboratory quality. Program activities achieved an average output per laboratory of 23±2 trainings of target personnel, 11±2 site visits, and 6027±2454 minutes of video-conference activity participation with quality mentors. Participating laboratories significantly improved performance in pre- and post-implementation audits as a program outcome over a 15-month evaluation period by a mean percent difference of 22±12% (p=0.002). A separate cross-sectional comparison found that 2019 overall scores of intervention laboratories (median=56.5) were significantly higher (p=0.0001) than those of a sample of laboratories with no intervention (median=22%). Significant differences between comparison groups were found in all individual audit sections addressed directly through program activities. In subsequent analysis of physician satisfaction ratings, corrected report frequencies, test turnaround times, and external quality assessment performance, changes in these measures of laboratory quality have not yet demonstrated the desired long-term outcome. This study demonstrates, however, the potential of the Cambodia laboratory information system and national external quality assessment program as long-term monitors of quality and provides recommendations for improved data collection and process improvement. Regarding the role of quality mentors, a test for correlation between pre- and post-audit score differences and laboratory personnel participation time in remote mentoring activities identified a strong monotonic relationship (rs=0.66, p=0.02). This relationship highlights the role of remote mentoring through video communication technologies as a potentially valuable approach for future program development in similar, resource-limited settings. The study as a whole shows that LQMS performance and conformity in participating laboratories has improved significantly in Cambodia after a mixture of remote and on-site mentoring along with a formal training curriculum, suggesting that replication of these approaches may be beneficial if expanded to non-intervention laboratories, which currently demonstrate significantly lower performance in LQMS audits.