This paper presents a method for recovering and identifying image regions from an initial oversegmentation using qualitative knowledge of its content. Compared to recent works favoring spatial information and quantitative techniques, our approach focuses on simple a priori qualitative inclusion and photometric relationships such as "region A is included in region B", "the intensity of region A is lower than the one of region B" or "regions A and B depict similar intensities" (photometric uncertainty). The proposed method is based on a two steps" inexact graph matching approach. The first step searches for the best subgraph isomorphism candidate between expected regions and a subset of regions resulting from the initial oversegmentation. Then, remaining segmented regions are progressively merged with appropriate already matched regions, while preserving the coherence with a priori declared relationships. Strengths and weaknesses of the method are studied on various images (grayscale and color), with various intial oversegmentation algorithms (k-means, meanshift, quickshift). Results show the potential of the method to recover, in a reasonable runtime, expected regions, a priori described in a qualitative manner. For further evaluation and comparison purposes, a Python opensource package implementing the method is provided, together with the specifically built experimental database.
11 May 2018 in Journal of the Optical Society of America A
The proposed approach exploits a priori known qualitative inclusion and photometric relationships between image regions, represented by oriented graphs. Our work assumes a sequential image segmentation procedure where regions are progressively segmented and recognized by associating them with corresponding nodes in graphs related to the prior knowledge. The main contribution concerns the parameterization of the -means clustering algorithm, to be used during the segmentation procedure, and the graph-matching-based identification of resulting clusters, corresponding to regions declared in graphs. The parameterization of -means is based on known relationships as well as on regions that have been segmented and recognized at previous steps. Parameters are the region of interest within which -means clustering is constrained, the number of clusters, and seeding constraints. Photometric relationships built from resulting clusters are matched with a priori known relationships to identify each cluster, this being formulated as an exact graph-matching problem. The potential of this approach is studied in four use cases involving real gray-scale and color images with dedicated sequential analysis procedures. Processing results are compared with those obtained without the proposed parameterization of -means, as well as with some other clustering approaches. Results show the relevance of our approach, in particular in terms of segmentation accuracy, computation time, and seeding reliability.
This paper addresses the fundamental task of semantic image segmentation by exploiting structural information (spatial relationships between image regions). To perform such task, we propose to combine a deep neural network (CNN) with inexact “many-to-one-or-none” graph matching where graphs encode efficiently class probabilities a nd structural information related to regions segmented by the CNN. In order to achieve node classification, a basic 2 -layer graph neural network (GNN) based on the edge-conditioned convolution operator (ECConv), managing both node and edge attributes, is considered. Prelim-inary experiments are performed on both a synthetic dataset and a public dataset of face images (FASSEG). Our approach is shown to be resilient to small training datasets that often limit the performance of deep learning thanks to a preprocessing task of graph coarsening. Results show that the proposal reaches a perfect accuracy on synthetic dataset and improves performance of the CNN by 6% (bounding box dice index) on FASSEG. Moreover, it enhances by 27% the initial Hausdorff distance (i.e. with CNN only) using the entire training dataset and by 41% with only 75% of training samples.