Method of Adding Color Information to Spatially-Enhanced, Bag-of-Visual-Words Models

Abstract

This thesis provides a late-fusing method, based on the HoNC (Histogram of Normalized Colors) descriptor, for combining color with shape in spatially-enhanced-BOVW models to improve predictive accuracy for image classification. The HoNC descriptor is a pure color descriptor that has several useful properties, including the ability to differentiate achromatic colors (e.g., white, grey, black), which are prevalent in real-world images, and to provide illumination intensity invariance. The method is distinguishable from prior late-fusing methods that utilize alternative descriptors, e.g., hue and color names descriptors, that are lacking with respect to one or both of these properties. The method is shown to boost the predictive accuracy by between about 1.9% - 3.2% for three different spatially-enhanced BOVW model types, selected for their suitability for real-time use cases, when tested against two datasets (i.e., Caltech101, Caltech256), across a range of vocabulary sizes. The method adds between about 150 – 190 mS to the models’ total inference time.