Use of Crowdsourcing to Diagnose Surgical Site Infections

Abstract

Background: Surgeons can use wound photographs to monitor for surgical site infections (SSI). Mobile Health apps that allow patients to securely transmit photos to providers for remote wound surveillance are available, but this new patient generated data stream demands a new monitoring process. Crowdsourcing (employing internet users to rapidly perform discrete tasks) has potential as a scalable method for wound triage. However, the ability of the crowd to assess and triage wound photographs is unknown. Methods: Ten case scenarios (6 SSI, 4 non-SSI), presented as 4-6 sequential days of surgical wound photographs and symptoms, were administered as a survey through Amazon Mechanical Turk, a global crowdsourcing platform. Participants provided demographics and prior experience with SSI. SSI was defined, but no additional diagnostic training was provided. For each scenario day, they indicated if they felt an SSI was present, level of confidence, and their triage recommendation for treatment or follow-up. Triage appropriateness was defined as escalation of care when SSI was suspected or non-escalation of care in the absence of SSI. SSI and non-SSI cases were analyzed to determine diagnostic accuracy and appropriateness of recommendations. Results: 1171 participants completed the survey within 6 hours. After quality control for survey completion, data from 993 participants (3311 cases) were analyzed. 530 (53%) of participants were female, mean age was 35.5 years (SD±11.38), and 741 (74.6%) had prior experience with surgical wounds. Overall diagnostic accuracy was 34.6%; 18.1% of SSIs and 59.6% of non-SSI were correctly identified. Personal history of SSI was associated with improved accuracy in both SSI (19.8% v 16.2%,p=0.037) and non-SSI cases (62.9% v 55.8%,p=0.008). Higher levels of confidence were associated with higher accuracy in non-SSI cases (OR 1.34 [1.28-1.43],p<0.001) and lower accuracy in SSI cases (OR 0.84 [0.80-0.86],p<0.001). Triage recommendations were correct in 52.8% of cases; 45.7% of non-SSI cases were over-triaged and 57.6% of SSI cases were under-triaged. Conclusions: Crowdsourcing without training had poor performance in the detection of SSI using wound photography. More work is required to establish standards for wound image features of SSI to develop triage training tools before crowdsourcing can be employed for SSI surveillance.