CARE - Clinician Augmented Reality Environment: Developing an Apple Vision Pro Framework for Image-guided Surgeries

Abstract

In the evolving landscape of surgical specialties, the shift toward minimally invasive procedures has become increasingly prominent, with most surgeries now being guided by imaging techniques such as laparoscopy, endoscopy, and fluoroscopy. Monitors have become essential for these procedures, yet their placement and visibility present significant challenges in terms of user experience and ergonomics, contributing to musculoskeletal disorders affecting 50-85% of practicing surgeons. Operating room staff continually seek innovations that simplify the environment, improve the surgical experience, and improve patient outcomes. Augmented Reality (AR) tools offer a promising solution by allowing optimal placement of virtual monitors, reducing the physical constraints and ergonomic challenges of traditional monitor placement. We are developing the Clinician Augmented Reality Environment (CARE) platform, a wireless streaming platform paired with an Apple Vision Pro AR headset software application, to enable wireless casting of any video source directly into the surgeon's field of view. Our implementation achieved streaming latencies under 71 milliseconds for 1080p video, meeting established requirements for real-time surgical applications. This thesis establishes a user-centered design methodology grounded in comprehensive interviews with 23 clinical and engineering stakeholders, analyzes the technical performance of our platform in surgical contexts, and evaluates the potential for clinical deployment and commercialization of AR-based surgical technologies.