Real-time Quantitative Assessment of Surgical Skill

Abstract

Faculty surgeons must accurately evaluate trainee performance in an apprenticeship training model in order to establish the skill level of their trainees and minimize their iatrogenic impact on patients. This is a subjective, resource-intensive process that has historically lacked quantitative rigor, with the exception of simple, summary measures such as task time. Moreover, the rise of minimally invasive laparoscopic surgical techniques has only increased the responsibilities of surgical faculty since it is more technically challenging and takes longer to master than traditional techniques. In the past two decades, technology has augmented surgical education with a number of simulators and robotic platforms. These technologies enable a more objective and automated evaluation of surgical skill since they provide quantitative data of the surgical act. This can, in turn, decrease the time, risk, and resource cost of training for students and faculty alike. One such platform—a laparoscopic box trainer called the Electronic Data Generation and Evaluation (EDGE) system—is herein used to collect a large, multi-institutional corpus of surgical data using the widely-adopted and heavily- validated Fundamentals of Laparoscopic Surgery (FLS) tasks. This corpus consists of subject demographics, tool motion data and corresponding video. A novel approach is proposed which seeks to find skill measures that apply universally across a variety of surgical procedures and enable skill evaluation in real-time using only tool motion data. Multiple criteria are used to rigorously define a set of “true experts” and additional skill categories. Vector quantization is used to extract relevant features from the surgical tool data and quantitative methods establish which features best discriminate skill. Hidden Markov models employ these features for temporal analysis capable of real-time feedback. Two criteria are adopted to determine whether these models provide value over simple task time. The models are shown to discriminate skill levels and provide significant value over simple task time. In contrast, the FLS scoring system is shown to provide no significant value over task time.