Mimicking Clinical Trials Using Real-World Data – A Novel Method and Applications

Abstract

Background Real-world data (RWD) have been widely used to evaluate the effectiveness and long-term safety of new treatments in practice to support evidence from clinical trials. Yet, the use of RWD would be limited when new treatments were slowly adopted in practice. Since clinical trial evidence may not be generalizable, it is important to know when and for whom the physicians would adopt the new treatments post clinical trials. Therefore, the first aim of this dissertation was to identify whether physician decisions on the adoption of treatment were influenced by the similarity of patient characteristics between trial participants and general target population. Additionally, RWD could be used to assess the generalizability of trials and possibly generate an external control group for a single arm study along with the propensity score-based approaches, which require individual-level trial data. When only summary data from trials are available, clinical trial samples can be simulated by using the observed correlations among baseline patient characteristics in the RWD. However, correlations found in RWD may not apply to trial data, and a different correlation structure among covariates would potentially bias the outcomes, especially if the outcomes data generating process includes effect modifiers or non-linear models. Therefore, using trial summary data and RWD, we aimed to develop an algorithm to approximate the true correlations of participants’ characteristics in a trial and generate trial data accordingly. Methods In Aim 1, we conducted an exploratory analysis using individual-level trial data and the SEER-Medicare linked database. The trial aimed to evaluate the efficacy of panitumumab plus FOLFIRI for treated metastatic colorectal cancer (mCRC), initially published in 2010. Using a propensity score approach, we estimated the likelihood of being enrolled in the trial given patient characteristics. Then, we observed the pattern of adopting any panitumumab-FOLFIRI regimen as the second-line therapy for the elderly patients with mCRC in practice, by the quartile-based groups determined by the probability of being enrolled in a trial. We also examined whether the pattern of adopting the regimen changed before and after 2010. The 95% confidence intervals (CI) of adopting the panitumumab-FOLFIRI regimen were obtained using the bootstrap method. In Aim 2, we developed an iterative algorithm using copula and resampling, which are based on the estimated propensity score for likelihood of enrollment in a trial given participants’ characteristics. Validation was performed using Monte-Carlo simulations under different scenarios where the marginal and joint distributions of covariates differ between trial samples and RWD. We also illustrated this method with two applications using an actual colorectal cancer trial and the SEER-Medicare linked database. First, we applied existing methods (i.e. standardized mean difference, B-index) to assess the generalizability of the trial. Then, we explored the feasibility of using RWD to generate an external control group by applying a parametric Weibull model trained in RWD to predict overall survival in the simulated trial cohort. The real-world target population were selected based on the eligibility trial criteria if applicable. Results In Aim1, among 2,815 treated metastatic colorectal cancer patients in the SEER-Medicare linked database, only 41 (1.5%) patients used any panitumumab-FOLFIRI regimen from 2008 to 2016. A positive association was observed between the probability of receiving the panitumumab-FOLFIRI regimen and the probability of being enrolled in the trial. After the trial results published in 2010, the probability of receiving any panitumumab-FOLFIRI regimen was 3% (95% CI: 0.011, 0.044) for patients who had higher probabilities to be enrolled in a trial (above the third quartile), whereas it was only 1 % (0.004, 0.016) for those who had lower probabilities (below the first quartile). By contrast, fewer than 1 % patients received any panitumumab-FOLFIRI regimen before 2010 regardless of the likelihood of being in a trial. The pattern of adopting the panitumumab-FOLFIRI regimen significantly differed before and after 2010 among patients whose probabilities of being enrolled in a trial were above the median. The mean difference in the probability of adopting the regimen before and after 2010 were 0.01 (0.002, 0.019) and 0.027 (0.015, 0.042) in the highest two quartiles respectively. Yet, the difference was very small because only few patients used the regimen in the real world. Additionally, we found that the treatment was adopted broadly many years later to patients whose characteristics were less similar with trial participants. In Aim 2, across all of the simulation scenarios, this iterative algorithm could successfully approximate correlations of covariates in a trial, which were closer to the true correlations than the correlations in RWD. The algorithm could also successfully reproduce the joint distribution of baseline characteristics for an actual cancer trial using summary data from the trial and the real-world cohort from the SEER-Medicare linked database. Therefore, we can obtain the similar estimates of standardized mean difference and B-index using simulated trial data (1.25 and 0.77 respectively) to assess the generalizability of the trial, comparing to the value of using the individual-level trial data (1.33 and 0.78 respectively). Additionally, no difference in approximated correlations was observed when we applied different eligibility trial criteria to select the target population. Lastly, we found that the adjusted survival estimates among the simulated trial population were close to the actual trial Kaplan-Meier (K-M) estimates, where the 95% confidence intervals of these survival curves were overlapped. Conclusions In this dissertation, we provided an approach for exploring how physicians would adopt the new treatment in practice post clinical trials using a specific colorectal cancer trial as an example in Aim1. Our study also implied that slowly adopting a new treatment in practice would lead to insufficient sample sizes and limit the use of RWD. Additionally, we found that our algorithm could be a feasible way to simulate individual-level clinical trial data when only summary data from the trials are available, which could be used to assess the generalizability of clinical trials given participants’ characteristics and inform decision making around the applicability of trial results to a real-world population.