Extracting information from clinical text with limited annotated data

Abstract

Electronic health record (EHR) data informs decision-making in clinical care; however, EHR data are generally underused for other purposes, including secondary use applications. The need to leverage EHR data, including clinical notes, is highlighted by the COVID-19 pandemic, as clinicians, researchers, and policymakers struggle to understand, treat, and contain a new disease. Secondary use cases for EHR data extend to many research areas related to healthcare effectiveness, epidemiology, and public health. Clinical notes contain many types of patient information that are not well characterized through structured data in the EHR, including social determinants of health (SDOH), symptoms, and other factors relevant to clinical informatics research. These patient data are frequently represented in the clinical narrative, rather than structured data, because structured data entry tools can be time-consuming and free-text entry allows richer descriptions. This text-encoded information can benefit secondary use applications, like large retrospective studies and clinical decision-support systems; however, the key information must first be automatically extracted, creating structured representations from unstructured clinical text. Data driven information extraction models require annotated data for training and evaluation, and annotated clinical data is limited by the high cost of annotation and privacy regulations. This work explores the automatic extraction of SDOH and COVID-19 diagnosis, testing, and symptom information from clinical text. The exploration of SDOH and COVID-19 focus on addressing the challenges associated with the limited availability of annotated clinical text. Here, "limited" is intended to mean a relatively small data set or low resource setting. The primary contributions of this work include the introduction of neural clinical information extraction models, new annotated clinical corpora, a novel active learning framework, and a secondary use application utilizing automatically extracted data. We present state-of-the-art neural information extraction approaches for SDOH and COVID-19 information, specifically designing the data-driven extraction architectures to achieve high performance with limited training data, by using multi-task learning and unsupervised pre-training. The extraction models generate event-based predictions that provide a detailed characterization of SDOH and COVID-19, achieving performance levels comparable to the inter-annotator agreement for several important factors. These information extraction approaches are relevant to a range of clinical data. As part of the exploration of SDOH and COVID-19, two new annotated corpora are developed: the Social History Annotation Corpus (SHAC) and the COVID-19 Annotated Clinical Text (CACT) Corpus. These corpora include detailed, high-quality annotations that characterize SDOH and COVID-19 across multiple dimensions. SHAC is unique in its annotation detail, size, and heterogeneity, and CACT is one of the first corpora with COVID-19 related annotations. These corpora are a substantial contribution to the available resources for training and evaluating machine learning-based extraction models at the University of Washington and for the larger clinical informatics community. In collecting SHAC, we introduced a novel active learning framework that uses a relatively simple text classification task as a proxy for a more complex event extraction task. The framework increased corpus richness and heterogeneity and improved extraction performance, relative to random selection. The largest performance improvements are associated with prominent risk factors, like drug and tobacco use, homelessness, and living with others. To demonstrate the utility of the automatically extracted data, this work presents a secondary use application exploring the prediction of COVID-19 infection. Incorporating automatically extracted symptom data improves COVID-19 infection prediction performance, beyond just using existing structured data.