Quantitative evaluation of image segmentation algorithms is crucial in the field of bioimage analysis. The most common assessment scores, however, are often misinterpreted and multiple definitions coexist with the same name. Here we present the ambiguities of evaluation metrics for segmentation algorithms and show how these misinterpretations can alter leaderboards of influential competitions. We also propose guidelines for how the currently existing problems could be tackled.
Nowadays, three dimensional (3D) cell cultures are widely used in the biological laboratories and several optical clearing approaches have been proposed to visualize individual cells in the deepest layers of cancer multicellular spheroids. However, defining the most appropriate clearing approach for the different cell lines is an open issue due to the lack of a gold standard quantitative metric. In this article, we describe and share a single-cell resolution 3D image dataset of human carcinoma spheroids imaged using a light-sheet fluorescence microscope. The dataset contains 90 multicellular cancer spheroids derived from 3 cell lines (i.e. T-47D, 5-8F, and Huh-7D12) and cleared with 5 different protocols, precisely ClearT, ClearT2, CUBIC, ScaleA2, and Sucrose. To evaluate image quality and light penetration depth of the cleared 3D samples, all the spheroids have been imaged under the same experimental conditions, labelling the nuclei with the DRAQ5 stain and using a Leica SP8 Digital LightSheet microscope. The clearing quality of this dataset was annotated by 10 independent experts and thus allows microscopy users to qualitatively compare the effects of different optical clearing protocols on different cell lines. It is also an optimal testbed to quantitatively assess different computational metrics evaluating the image quality in the deepest layers of the spheroids.
Mitosis detection represents a critical task in the field of digital pathology, as determination of the mitotic index (MI) plays an important role in the tumor grading and prognostic assessment of patients. Manual determination of MI is a labor-intensive and time-consuming task for practitioners with rather high interobserver variability, thus, automation has become a priority. There has been substantial progress towards creating robust mitosis detection algorithms in recent years, primarily driven by the Mitosis Domain Generalization (MIDOG) challenges. In parallel, there has been growing interest in the molecular characterization of mitosis with the goal of achieving a more comprehensive understanding of its underlying mechanisms in a subphase-specific manner. Here, we introduce a new mitotic figure dataset annotated with subphase information based on the MIDOG++ dataset as well as a previously unrepresented tumor domain to enhance the diversity and applicability of the dataset. We envision a new perspective for domain generalization by improving the performance of models with subtyping mitotic cells into the 5 main stages of normal mitosis, complemented with an atypical mitotic class. We believe that our work broadens the horizon in digital pathology: subtyping information could provide useful help for mitosis detection, while also providing promising new directions in answering biological questions, such as molecular analysis of the subphases on a single cell level.