Do hospital performance rankings sufficiently account for underlying patient risk? The value of information in outcomes-based risk adjustment

Abstract

Objective: To determine the optimal amount of information that should be included in a risk adjustment model as it pertains to health care performance based financing. Data Sources: Health Care Cost and Utilization Project (HCUP) State Inpatient Databases (SID) for New York state 2005-2009 Study Design: Replicated existing hierarchical logistic risk adjustment models for mortality and readmission on a large administrative dataset of patients with a primary diagnosis of acute myocardial infarction (AMI), heart failure (HF), or pneumonia (PN). Machine learning techniques were also applied to incorporate individual patient diagnoses as discrete predictors. All models were run on identical patient populations and evaluated using cross-validation along with comparison of final facility rankings. Principal Findings: The c-statistic for 30-day mortality using individual 5-digit ICD-9 diagnoses as predictors was .80 for AMI, .76 for HF, and .78 for PN, compared to .75, .73, and .74, respectively for Centers for Medicare and Medicaid Services (CMS) models. Similar improvements were observed for in-facility mortality, however not for 30-day readmission. Conclusions: Facility performance rankings could be refined by including more patient information, however the marginal return on information appears to be low with CMS models as the point of reference.