Vision based Analysis in Fishery Applications

Abstract

Vision-based analyses have drawn increasing attention since they enable a non-extractive and non-lethal approach to fisheries survey. The challenges of extracting information from vast amounts of video or image data can be reduced by using automatic analysis techniques for segmentation, tracking, length/size measurement and species identification. The goal of the work is therefore to build a successful automatic development of these algorithms, which will greatly ease one of the most onerous steps in vision-based fisheries survey. One of the major fishery applications is built for on-board chute-based images, related to fish species identification, segmentation and length/size measurement. Recently, image classification and recognition have been well studied in computer vision society with the rapid development of convolutional neural networks. However, there are still some challenges for species identification in fishery applications. For example, the fisheries surveys are conducted each year. Due to the environment change, the data acquired for each year may have large difference. As we know, for supervised learning method, if the testing data cannot be well represented by the training data, then the performance usually degrades on the testing set. On the other side, it is usually expensive to label all the data for each year. To address such issues, species identification with active learning approach is proposed for efficient labeling and model adaptation. We iteratively select informative samples for human labeling. The sample selection is formulated as a sparse modeling problem and efficient approximation solutions are proposed. Another application on chute image dataset is fish size and length measurement which requires a reliable segmentation approach. However, there are two major challenges about the segmentation. One is that images may be blurred due to the spray of water on the camera lens, and the other is that some part of the fish body is out of the camera view. We present an innovative and effective contour-based segmentation refinement method to address such problems. The other part of our completed system is fish abundance estimation based on the underwater videos from a remotely-operated vehicle (ROV). Once the video is captured, the live fish can be efficiently tracked and counted with our proposed system. The tracking and counting algorithm is followed by tracking-by-detection framework. The live fish are first detected by offline trained fish detector and TrackletNet tracker (TNT) is further developed for counting purpose. However, due to the diversity of fish poses, the deformation of fish body shape and the color similarity between fish and background, the detection performance greatly degrades, resulting in large error in tracking and counting. To deal with such issue, tracklet classifier is built for removing false positives, while TNT is trained to compensate the missing detections and ID switches. Finally, the tracking is conducted by a graph clustering method. This proposed strategy effectively addresses the false detection, occlusion, ID switches and largely decreases the tracking error.