MIXED METHODS CURRICULUM EVALUATION: MATERNITY CARE COMPETENCE

Abstract

Purpose: This study aims to pilot a mixed method design for residency curriculum evaluation using the Madigan Family Medicine (FM) Residency's Maternity Care Curriculum. The actual curriculum evaluation purposes to assess the FM residency's achievement of its curriculum objectives and key components established by the 2007 and 2011 ACGME Program Requirements for FM1. It is an Institutional Review Board (IRB) exempt study approved by the Madigan Department of Clinical Investigation and the University Of Washington School Of Public Health IRB. It fulfills the requirement for Masters of Public Health thesis and master's degree. The ACGME Program Requirements are currently in the process of modification. The Madigan FM Residency leadership was concerned that residents might have difficulty achieving adequate obstetrical volume under the new requirement. Methods: This residency curriculum evaluation used a mixed methods (quantitative and qualitative) design. The quantitative method consisted of a retrospective review and analysis of previously existing data from a national database (MyEvaluations) that described the number of deliveries completed by current residents (2013-2015 graduates) and recent graduates from 2006-2012, as well as rotations completed by the 2012-2015 graduates. The "overall" and "female specific" sections on the 2007-2012 In Training Exams (ITE) were compared for significant trends. The qualitative research was accomplished through semi-structured interviews conducted from December 2012 through February 2013 with 22 current and recently graduated residents. Each interview consisted of responses to thirteen standardized questions. Interviews were audio recorded, de-identified, and transcribed. The qualitative analysis employed qualitative software for assisting with coding and identification of clusters of meaning. These clusters of meaning were then used for thematic analysis, which described the residents' experience of the curriculum, as well as their competence development in the curriculum. Results: The overall mean spontaneous vaginal deliveries (SVD) for Graduates 2006-2013 was 63 per resident. The mean SVD was not significantly different among the graduate years (p =0.297), but the mean number of C-sections was significantly different among the graduate years (p=0.005). The FM residents graduating in 2013 completed an average of four Maternity Care rotations (one L&D in PGY1, one L&D in PGY2, one complicated OB (COB) in PGY2, and one L&D in PGY3). Overall Training Exam (ITE) scores, which were analyzed among graduate years for revealed significant differences in the 2007, 2008, and 2009, (p=0.008, 0.003, and <0.0001 respectively). When class trends were observed in the Maternity Care specific section, the scores noted on the 2011 and 2012 ITE revealed an increase in scores in both the 2013 and 2014 graduate classes, but not a difference that reached significance. Four of the five graduates from 2012 were interviewed. Zero out of the four was practicing continuity obstetrics. One out the four saw obstetric patients in clinic. The qualitative analysis described themes in Maternity Care competence in obstetrics and the residents' experience in the curriculum. Conclusions: The information revealed through the resident interviews and thematic analysis provided valuable insights that will help influence the ongoing evolution of the FM Program Requirements in Maternity Care for the Madigan FM Residency. A Mixed Method design provides a richer evaluation of curricula than a pure quantitative approach. The Curriculum Evaluation revealed that the numbers of deliveries and rotations (exposures) are compliant with ACGME 2007 (2011) Program Requirements. Maternity Care specific ITE scores improved over the curriculum in the cohort of residents. Perceived and measured competence was achieved by the Madigan FM residents through the current Maternity Care curriculum.