Superpixel segmentation consists of partitioning images into regions composed of similar and connected pixels. Its methods have been widely used in many computer vision applications, since it allows for reducing the workload, removing redundant information, and preserving regions with meaningful features. Due to the rapid progress in this area, the literature fails to catch up on more recent works among the compared ones and to categorize the methods according to all existing strategies. This work fills this gap by presenting a comprehensive review with a new taxonomy for superpixel segmentation, in which methods are classified according to their processing steps and processing levels of image features. We revisit the recent and popular literature according to our taxonomy and evaluate 23 strategies and a grid segmentation baseline based on nine criteria: connectivity, compactness, delineation, control over the number of superpixels, color homogeneity, robustness, running time, stability, and visual quality. Our experiments show the trends of each approach in superpixel segmentation and discuss individual trade-offs. Finally, we provide a new benchmark for superpixel assessment, available at https://github.com/IMScience-PPGINF-PucMinas/superpixel-benchmark .
As constituent parts of image objects, superpixels can improve several higher-level operations. However, image segmentation methods might have their accuracy severely compromised for reduced numbers of superpixels. To mitigate the problem, we introduce Dynamic Iterative Spanning Forest (DISF), a seed-based method that improves all components in the Iterative Spanning Forest (ISF) framework for superpixel segmentation. DISF relies on a new strategy for seed estimation that can find more relevant seeds, reconstruct relevant edges along with iterations, and guarantee the desired number of superpixels. DISF also assures optimal spanning forests for path costs based on dynamic arc-weight estimation, being faster as the desired number of superpixels grows. We show that DISF can improve effectiveness on three datasets with distinct object properties, requiring significantly fewer iterations than all seed-based baselines.
This article aims to understand the practical features of hierarchies of morphological segmentations, namely the quasi-flat zones hierarchy and watershed hierarchies, and to evaluate their potential in the context of natural image analysis. We propose a novel evaluation framework for hierarchies of partitions designed to capture various aspects of those representations: precision of their regions and contours, possibility to extract high quality horizontal cuts and optimal non-horizontal cuts for image segmentation, and ease of finding a set of regions representing a semantic object. This framework is used to assess and to optimize hierarchies with respect to the possible pre-and post-processing steps. We show that, used in conjunction with a state-of-the-art contour detector, watershed hierarchies are competitive with complex state-of-the-art methods for hierarchy construction. In particular, the proposed framework allows us to identify a watershed hierarchy based on a novel extinction value, the number of parent nodes, that outperforms the other hierarchies of morphological segmentations. This coupled with the fact that watershed hierarchies satisfy clear global optimality properties and can be efficiently computed on large data, make them valuable candidates for various computer vision tasks.