Characterizing Robot Vision Solutions for Anomaly Detection in Confined Spaces

Abstract

Industrial structures are often susceptible to damage due to environmental and human factors and need to be inspected to determine safety, longevity, damage repair requirements and strength of the structure. Confined spaces in mechanical structures contain featureless surfaces hindering visual recognition, and are conventionally inspected manually implementing a labor and cost intensive process, subjecting the inspectors to a prolonged exposure to dark, hazardous, stifling, tight spaces. We have developed a robotic framework for a ground robot enabled with vision sensors to perform scanning, and inspection of confined spaces in order to perform anomaly detection and anomaly classification. This research addresses unique anomalies such as rust patches and unconventional objects like industrial maintenance tools, and proposes a novel approach to evaluating object detection and classification by fusing RGB and Depth information in a computationally efficient technique. This study highlights the effect of vision sensor performance, controlled illumination, robot pose with respect to anomaly, integration of color and depth input, and the use of images of sections in a confined space, for anomaly recognition. The proposed system employs transfer learning, utilizing a partially pre-trained YOLOv8 model on a custom dataset of anomaly images and getting a result of mean Average Precision (at IoU50) = 0.83 for industrial tools and a mean average precision (at Iou50) = 0.54 for rust anomalies.