Training-Free Feature-Based Ranking for Multi-Candidate Mask Selection in Image Segmentation

Abstract

Recent advances in neural-network-based and promptable image segmentation have improved the accuracy, robustness, and flexibility of object-mask generation. Under ambiguous visual conditions, however, these systems may produce multiple plausible candidate masks for the same prompted target, whereas downstream applications generally expect only one mask. This creates a candidate-selection problem: determining which candidate most accurately delineates the intended target. Existing approaches often rely on learned quality predictors, additional supervision, calibration data, reference datasets, or model-specific modification. These dependencies increase implementation effort and limit portability when the candidate image segmentation system is pretrained, externally supplied, or otherwise fixed.This thesis presents the ACES (Area, Center-location, Edge-proximity, and Silhouette) framework, a training-free, user-guided scoring system for inference-time candidate ranking. Inspired by classical binary-image analysis, ACES measures four corresponding geometric and morphological properties for each candidate mask and compares these observations with four target reference values representing the expected properties of the intended target. These comparisons produce four compatibility terms, which are summed to obtain the final ACES score. The highest-scoring candidate mask is recommended as the final output. The ACES framework provides a versatile default set of target reference values derived through controlled experiments. These reference values can be modified based on application knowledge and the characteristics of the target geometry. An existing candidate set can therefore be re-ranked under different target references without repeating candidate mask generation. Since the ranking requires only candidate masks and image dimensions, the scoring computation can be applied to outputs from different segmentation pipelines as a lightweight post-processing stage. The ACES framework is evaluated on the DAVIS 2016 benchmark using candidate masks generated by a Grounding DINO and SAM 2 pipeline. Two settings are examined. The first applies the default reference values to all sequences without target-specific adjustments. The second derives reference values based on the geometric characteristics of the intended target. In the default setting, ACES increases mean intersection over union (IoU) from 0.6175 for upstream confidence-based selection to 0.7308. The framework is most effective when the candidate set contains at least one mask that substantially overlaps the intended target and when the target’s scale, location, and shape remain sufficiently stable across the evaluated frames to be represented by a single set of reference values. Under these conditions, ACES can suppress distractor masks that receive higher upstream confidence and recommend a candidate that more closely matches the intended target. These results demonstrate that reference-based geometric scoring provides an effective, interpretable, and portable basis for multi-candidate mask selection while preserving the existing segmentation pipeline.