11 June 2026 in IEEE Open Journal of Signal Processing
Reliable instance segmentation of cells in microscopy images is essential for quantitative imaging analysis, yet estimating segmentation quality on unseen data, though fundamental, remains largely overlooked. In practice, performance assessment typically requires manual ground-truth annotations, which are costly and impractical at scale. We address this evaluation gap by introducing QANet, a neural framework for post-hoc quality assessment of instance segmentations. Unlike segmentation models, QANet does not predict masks; instead, it receives an image and a segmentation produced by any method and estimates a quantitative quality score that approximates standard evaluation metrics, without requiring ground-truth annotations at inference time. QANet is model-agnostic and formulated as a regression task over image–mask pairs. It is built on the RibCage architecture, which performs multi-scale comparison between image content and segmentation structure, enabling sensitivity to both global shape consistency and fine boundary errors. Training is performed using synthetically perturbed segmentations with known quality scores, enabling supervision across controlled error modes while eliminating the need for large annotated failure datasets. We evaluate QANet on 2D and 3D datasets from the Cell Segmentation Benchmark and demonstrate accurate prediction of both overlap-based measures and boundary-sensitive metrics across multiple segmentation methods. The code is available at https://github.com/amita1996/QANet.