Flexible and Robust Treatments for Missing Traffic Sensor Data

Abstract

The focus of the work contained in this thesis is missing data treatments in traffic loop detector datasets. This work is motivated by the need to improve data quality and coverage for performance reporting and system management decisions. Missing data, whether due to hardware malfunction or error detection and removal, is a critical concern in loop detector data quality control in Washington State and elsewhere, and can quickly become the controlling factor in overall data quality as the rate of missingness increases. First, the various causal factors and resulting patterns of missingness in loop detector datasets are discussed with respect to the assumptions underlying common missing data treatments. Next, two multiple imputation methodologies are introduced for loop detector data, which have seen use in a number of fields but have not yet been applied to traffic data. These methods are able to take advantage of the various spatial correlation structures present in volume and speed data, and can produce reliable imputation even under high rates of missingness and missing entire days and months. The proposed imputation algorithms are demonstrated in different locations, time periods, and missing data patterns, and are shown to be capable of reliably representing the statistical properties of the true data. Aggregation levels, model structure, and limitations of the proposed methods are discussed, and some guidelines for implementation are presented. The proposed algorithms are designed to be incorporated into a comprehensive quality control process for traffic data, to be implemented as part of the STAR Lab DRIVE Net data analysis, visualization, and dissemination platform.