Towards Trustworthy Modeling of Patient Trajectory with Longitudinal Electronic Health Records

Abstract

Patient trajectory modeling, which predicts future clinical events using data from longitudinal electronic health records (EHRs), is expected to be of value for personalizing disease management. Yet the adoption of powerful deep learning (DL) models is often hindered by their "black-box" nature, creating a barrier to clinical trust. This challenge is compounded when modeling complex temporal dependencies between lab test results, treatments, and clinical events in the EHR, and is further exacerbated by the persistent difficulty these models face in generalizing across diverse patient populations, varying data elements, and different disease states.This dissertation develops novel interpretable and generalizable frameworks for patient trajectory modeling, motivated by the hypothesis that models that account for the full dynamics of a patient’s history will produce more reliable predictions than simpler models, and that these predictions can also be made transparent and robust across diverse clinical contexts. The work is structured around four complementary aims that collectively affirm this hypothesis while innovating in terms of both deep learning methods and interpretable learning tools. It begins by developing an interpretable and generalizable deep learning framework for predicting survival in metastatic prostate cancer from pre-metastasis serial PSA values and treatments, establishing the value of trajectory-based modeling in a focused clinical setting. Building on this foundation, the work then advances from discrete-time modeling to a more precise continuous-time framework by introducing a model that learns continuous latent trajectories and uses a divide-and-conquer interpretation to explain how clinical changes drive outcomes. To broaden generalizability beyond training task-specific models, the dissertation next develops a multi-agent system that leverages a chain of large language model (LLM) agents with a long-term memory to reason over long, noisy, and heterogeneous EHR data for zero-shot cancer early detection. Finally, the work enables the self-evolving capability of this multi-agent system through an evolving experience pool and multi-agent reinforcement learning for lung cancer early detection, allowing the system to continuously adapt to new patient cohorts. Through these complementary aims, this research traces an arc from task-specific prediction to generalizable and self-evolving reasoning, powered by DL-based sequential models and LLM-based systems. It shows that faithfully modeling the temporal information in patient histories can make the predictions accurate, robust, and interpretable, with interpretability and generalizability advancing together rather than in tension. In doing so, this dissertation seeks to contribute to the development of more trustworthy AI tools that can support personalized clinical decision-making across a spectrum of complex medical domains.