Assessing the Concordance Between AI and Oncologist Treatment Recommendations in Breast Cancer Care: A Blinded Validation Study at Kenyatta National Hospital

Abstract

Artificial intelligence (AI) has emerged as a promising tool in oncology, with the potential to support clinical decision-making and expand access to evidence-based care. This study examines the clinical validity of Gukiza, a proprietary AI platform developed by Hurone AI for cancer management. For this study, a total of 21 de-identified breast cancer patients from Kenyatta National Hosptial were assessed. Treatment plans were created by Gukiza and a team of oncologists, and evaluated by an expert panel, as well as against National Comprehensive Cancer Network (NCCN) Clinical Guidelines. Results showed that AI-generated treatment plans were not significantly correlated with oncologist-generated plans, potentially reflecting limitations in the scoring framework, restricted variability in responses, or underlying differences in clinical reasoning. Although clinician-generated plans received a higher proportion of "Acceptable" and "Exceptional" ratings, this difference was not statistically significant. In contrast, an AI-assisted analysis of AI-generated plans demonstrated strong concordance with NCCN guidelines, highlighting their potential alignment with evidence-based standards despite limited agreement with clinician ratings. Subgroup analyses indicated that postmenopausal status was associated with lower odds of treatment plan unacceptability, while the presence of comorbidities was associated with higher odds, although neither association reached statistical significance. These findings suggest that AI-generated treatment recommendations are largely NCCN guideline-adherent, and while the data from the blinded concordance assessment did not show significant correlation, the analysis highlights some of inherent challenges in comparing AI tools with clinical standards of care. Further validation in larger, more diverse patient populations is needed.