Surgical Scene Understanding Towards Human-Centered Collaboration in Robotic Surgery

Abstract

Robot-assisted minimally invasive surgery brings together the expertise of highly-skilled surgeons with the increased precision and dexterity of assistive robots. Surgical robots currently in deployment provide surgeons with enhanced visualization, filter out hand tremors, and incorporate data integration and analytics to facilitate improved patient care. Mostsurgical robots, however, are still constrained to being teleoperated by a surgeon, with limited advances towards integrating automation in the operating room. The use of automation in surgery has the potential to extend the current benefits of robotic surgery to integrate more consistent care by reducing variability. While studies have explored automating tasks performed by a surgeon, the role of the first assistant during surgeries has often been underappreciated in the context of automation. However, this role is pivotal in ensuring the smooth progression of procedures, maintaining a clear surgical field, and providing critical support to the primary surgeon. To this end, offloading some sub-tasks performed by the first assistant to automated systems can reduce the risk of human error and optimize re- source allocation. Building on the viability of camera sensors as a reliable input, the goal of this thesis was to aid the development of an autonomous suction assistance tool by developing the infrastructure for surgical scene understanding. This work developed surgical segmentation models for efficient segmentation of binary, parts and instrument segmentation, established approaches for addressing the paucity of labeled data in surgical settings, and developed a framework to benchmark approaches to cooperative autonomy through the lens of an assistive suction task