Searching for Predictive Subgroups with Enhanced Treatment Effect in Clinical Trials using SHAPES

Abstract

The complexity of the human genome and variability of individual health histories, living environments, and lifestyles have motivated the development of precision medicine with the goal to enable health care providers to tailor treatments to each patients’ unique characteristics. Targeted delivery of treatments/preventative strategies to subgroups who benefit sufficiently or solely can optimize treatments and improve the benefit/risk ratio for many patients. Therefore, effective methods of identifying subgroups with sufficiently large benefit are highly desired in clinical trials. However, the number of factors under consideration to define subgroups can be large and performing multiple tests to identify subgroups requires appropriate control of the overall Type I error. Here, we focus on investigating the Type I error rate for a new approach, SHAPES, which was recently developed by Prince (2015). SHAPES is an approach to search for the subgroups by directly restricting the type of subgroups that will be accepted. SHAPES subgroups must be convexly or co-convexly connected in the Boolean space. Prince explored the performance of SHAPES in terms of Type I error control and power under various scenarios considering up to four covariates for subgroup definition. We expand the evaluation of Type I error rate and the settings for evaluation as follows: 1) increasing the total number of covariates considered for subgroup definition from 4 up to 50; 2) increasing the maximum number of covariates that can be used in the definition of a subgroup from 4 up to 6; 3) considering both independent and correlated covariates; 4) considering the prevalence of all covariates to be either 0.5 or 0.2. To evaluate the Type I error rate of SHAPES, we simulate data under different scenarios. Briefly, two-arm randomized trials with either binary or continuous outcomes and various number of binary covariates are simulated under the null, i.e. in all simulations it is assumed that in truth there is no treatment benefit for the study population overall or for any subgroup. Qualified SHAPES subgroups are listed, and stratification models (for selected groups) as well as interaction models (for full population) are fitted. We obtain critical values from 5,000 (and up to 15,000 for selected scenarios) simulations under the null hypothesis where all the covariates are generated to be mutually independent. The critical values are then used in another 5,000 (and up to 15,000 for selected scenarios) simulations with independent or correlated covariates to calculate the Type I error rate. The number of covariates under consideration here is limited by the amount of time required for the simulations. When the covariates are independent, the overall Type I error of SHAPES is maintained around the pre-specified α level and appears very robust for the scenarios we simulate. No monotonic trend is observed as the number of covariates considered for subgroup definition increases (up to 50) or the number of covariates for subgroup definition increases (up to 6). There is no obvious difference between the results of the stratification models and the interaction models. However, when some or all of the covariates are correlated, the overall Type I error is not as consistently close to the nominal α level as in the independent scenarios. For a binary endpoint with correlated covariates, the Type I error rate is slightly higher than the pre-specified α level, while for a continuous endpoint, the overall Type I error decreases as the correlation between covariates increases and in some scenarios is significantly lower than the nominal level. In summary, for most of the simulated scenarios the Type I error rate of SHAPES was close or reasonably close to the nominal rate for both binary and continuous outcomes as well as independent and correlated covariates. Future research might focus on reducing the time required for the simulations to allow investigation of larger number of covariates.