Xiaoyi Ji

Xiaoyi Ji

Auto-generated profile, based on publicly available information

Xiaoyi Ji

Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
24 December 2025 in BMJ Open
Verified
Journal Article
Open access
XJXiaoyi Ji
Xiaoyi Ji
RZRenata Zelic
Renata Zelic
OAOskar Aspegren
Oskar Aspegren
NMNita Mulliqi
Nita Mulliqi
MFMichelangelo Fiorentino
Michelangelo Fiorentino
KKKimmo Kartasalo
Kimmo Kartasalo
Introduction: Prostate cancer diagnosis and treatment planning depend on accurate histopathological assessment of needle biopsies, particularly through the Gleason scoring system. The inherently subjective nature of the grading creates variability between pathologists, potentially resulting in suboptimal patient management decisions. These reproducibility challenges extend beyond Gleason scoring to encompass other critical diagnostic and prognostic markers, including cancer volume quantification and detection of cribriform morphology patterns and perineural invasion. Artificial intelligence (AI) applications in digital pathology have emerged as promising solutions for enhancing diagnostic consistency and accuracy, with recent research demonstrating that automated systems can match expert-level performance in prostate biopsy evaluation. Nevertheless, comprehensive validation studies have revealed concerning limitations in model generalisability when deployed across different clinical environments and patient populations. Recent systematic reviews revealed widespread risk-of-bias limitations and insufficient external validation in AI diagnostic studies, highlighting critical needs for accumulated evidence supporting generalisability before clinical implementation. Rigorous external validation with preregistered protocols using independent datasets from diverse clinical settings remains essential to establish the reliability and safety of AI-assisted prostate pathology systems. Methods and analysis: This study protocol establishes a framework for the retrospective external validation of an AI system developed for prostate biopsy assessment, to be conducted on the case-control samples of the National Prostate Cancer Register of Sweden, ProMort study (1998-2015). The primary aim is to evaluate the AI model’s diagnostic accuracy and Gleason grading performance using completely independent datasets separate from any model development or previously used validation cohorts. The diversity of the validation samples, spanning multiple geographic regions, temporal collection periods and reference standards, allows evaluation of model robustness across varied clinical contexts. Secondary aims encompass evaluating AI performance in cancer length estimation and detection of cribriform patterns and perineural invasion. This protocol delineates procedures for data collection, reference standard clarification and prespecified statistical analyses, ensuring comprehensive validation and reliable performance assessment. The study design conforms to established reporting guidelines Checklist for Artificial Intelligence in Medical Imaging (CLAIM) and Standards for Reporting Diagnostic Accuracy Studies using Artificial Intelligence (STARD-AI), and recognised best practices for AI validation in medical imaging. Ethics and dissemination: Data collection and usage were approved by the Swedish Regional Ethics Review Board and the Swedish Ethical Review Authority (permits 2012/1586-31/1, 2016/613-31/2, 2019-01395, 2019-05220). The study adheres to the Declaration of Helsinki principles, and findings will be made available in open access peer-reviewed publications.

23 September 2025 in medRxiv
Verified
Preprint
Open access
PDF file
XJXiaoyi Ji
Xiaoyi Ji
RZRenata Zelic
Renata Zelic
OAOskar Aspegren
Oskar Aspegren
NMNita Mulliqi
Nita Mulliqi
Michelangelo FiorentinoMichelangelo Fiorentino
Michelangelo Fiorentino
KKKimmo Kartasalo
Kimmo Kartasalo
Introduction: Prostate cancer diagnosis and treatment planning depend on accurate histopathological assessment of needle biopsies, particularly through the Gleason scoring system. The inherently subjective nature of the grading creates variability between pathologists, potentially resulting in suboptimal patient management decisions. These reproducibility challenges extend beyond Gleason scoring to encompass other critical diagnostic and prognostic markers, including cancer volume quantification and detection of cribriform morphology patterns and perineural invasion. Artificial intelligence (AI) applications in digital pathology have emerged as promising solutions for enhancing diagnostic consistency and accuracy, with recent research demonstrating that automated systems can match expert-level performance in prostate biopsy evaluation. Nevertheless, comprehensive validation studies have revealed concerning limitations in model generalisability when deployed across different clinical environments and patient populations. Recent systematic reviews revealed widespread risk-of-bias limitations and insufficient external validation in AI diagnostic studies, highlighting critical needs for accumulated evidence supporting generalisability before clinical implementation. Rigorous external validation with preregistered protocols using independent datasets from diverse clinical settings remains essential to establish the reliability and safety of AI-assisted prostate pathology systems. Methods and analysis: This study protocol establishes a framework for the retrospective external validation of an AI system developed for prostate biopsy assessment, to be conducted on the case-control samples of the National Prostate Cancer Register of Sweden, ProMort study (1998-2015). The primary aim is to evaluate the AI model’s diagnostic accuracy and Gleason grading performance using completely independent datasets separate from any model development or previously used validation cohorts. The diversity of the validation samples, spanning multiple geographic regions, temporal collection periods, and reference standards, allows evaluation of model robustness across varied clinical contexts. Secondary aims encompass evaluating AI performance in cancer length estimation and detection of cribriform patterns and perineural invasion. This protocol delineates procedures for data collection, reference standard clarification, and prespecified statistical analyses, ensuring comprehensive validation and reliable performance assessment. The study design conforms to established reporting guidelines CLAIM and STARD-AI, and recognised best practices for AI validation in medical imaging. Ethics and dissemination: Data collection and usage were approved by the Swedish Regional Ethics Review Board and the Swedish Ethical Review Authority (permits 2012/1586-31/1, 2016/613-31/2, 2019-01395, 2019-05220). The study adheres to the Declaration of Helsinki principles, and findings will be made available in open access peer-reviewed publications. STRENGTHS AND LIMITATIONS: This study incorporates case-control subsamples from Sweden’s largest clinical prostate cancer database (the National Prostate Cancer Register, NPCR), capturing a broad spectrum of variation across Swedish regions. The validation dataset encompasses samples collected from 1998 to 2015, representing one of the first AI validation studies to systematically evaluate performance across such an extensive temporal range, capturing evolving histological sample preparation techniques and changing population characteristics. A consistent scanning and annotation platform during digitisation eliminates equipment-related technical variation, while standardised annotation protocols among pathologists ensure traceable and reliable reference standards. Case-control design with 50% cancer-related mortality may create spectrum and prevalence bias, limiting comparison with typical clinical populations and other AI studies. Differences between the diagnostic reporting guidelines applied to the AI model’s training data and our validation dataset may introduce systematic differences that affect the interpretation of AI-pathologist concordance measurements.

8 May 2026
Verified
Preprint
Open access
XJXiaoyi Ji
Xiaoyi Ji
Artificial intelligence (Al) has reached expert-level performance on curated benchmarks for prostate cancer histopathology, yet a gap persists before clinically trustworthy implementation, which requires more than accuracy in model validation. Real-world deployment introduces sources of variation that current development and evaluation designs may have overlooked, including differences across clinical sites, imaging devices, and data characteristics introduced by time. The extent to which contemporary AI systems keep reliable under such conditions, and the mechanisms by which their robustness can be assured, remain to be fully characterized. This thesis aims to investigate the generalization and robustness of AI models for computational prostate cancer pathology across multiple sources of variation and to examine possible strategies for improvement.In Study I, we addressed between-site variation and offered an AI model-agnostic method for color calibration of whole slide image (WSI) scanners. Using a biomaterial-based physical calibrant slide and a chosen color profile reference, cross-scanner generalization of cancer diagnosis and grading performance substantially improved for both a fully supervised convolutional model and a weakly supervised foundation model-based system across multiple cohorts. For example, the concordance of grading with pathologists improved from 0.354 to 0.738 for the internal cohort and from 0.439 to 0.619 for an external cohort with the fully supervised model. The method also outperformed existing computational color normalization approaches, while requiring no additional training data and introducing no visible artifacts.In Study II, we illustrated a secondary clinical application of AI-assisted grading. When a sensitivity-prioritized threshold was picked, an end-to-end attention-based multiple instance learning (ABMIL) model without retraining managed to reduce unnecessary immunohistochemistry orders by 20-44% across three cohorts of diagnostically challenging biopsies without missing any cancer, supporting the broader clinical utility of grading models in streamlining decision-making in prostate pathology and alleviating resource burdens.In Study III, we established a preregistered external validation protocol conforming to STARD-AI, defining data inclusion and exclusion criteria, reference standards, and performance evaluation metrics, while being transparent about the model architecture and data preprocessing pipeline, to enable a rigorous generalization assessment of the AI model on long-term archival material.In Study IV, we executed the protocol from Study III on 10,366 biopsy cores from 1,028 patients archived across 14 Swedish regions between 1998 and 2015. Core-level grading of the model presented comparable performance to several experienced pathologists, with an overall quadratic-weighted kappa of 0.86 against the reference standard. Confusion matrices and survival analysis further supported the model's ability to capture clinically relevant features from tissue morphology. Notably, the model performance remained stable across the 17-year archival span, superior in consistency to the foundation model-based systems.In Study V, we conducted the first investigation of temporal variation within a single scanner on contemporary AI computational pathology models. Evaluating models on WSIs obtained by rescanning a fixed biopsy set every 14 days over one year, we found that foundation model encoders carry strong temporal signatures from their pretraining that task-specific training of the downstream attention aggregator could only partially suppress, even with a training dataset exceeding 60,000 WSIs. In contrast, the end-to-end ABMIL model learned to focus mostly on morphological information and largely ignored irrelevant temporal variation. This representation-level difference was reflected in performance stability, as the end-to-end model maintained consistent self-agreement across the year, whereas several foundation model-based systems showed measurable degradation over the time span.List of scientific papersI. Xiaoyi Ji, Richard Salmon, Nita Mulliqi, Umair Khan, Yinxi Wang, Anders Blilie, Henrik Olsson, Bodil Ginnerup Pedersen, Karina Dalsgaard Sørensen, Benedicte Parm Ulhøi, Svein R. Kjosavik, Emilius A.M. Janssen, Mattias Rantalainen, Lars Egevad, Pekka Ruusuvuori, Martin Eklund, Kimmo Kartasalo. Physical color calibration of digital pathology scanners for robust artificial intelligence-assisted cancer diagnosis. Modern Pathology, 2025, 38(5): 100715. https://doi.org/10.1016/j.modpat.2025.100715II. Anders Blilie*, Nita Mulliqi*, Xiaoyi Ji, Kelvin Szolnoky, Sol Erika Boman, Matteo Titus, Geraldine Martinez Gonzalez, José Asenjo, Marcello Gambacorta, Paolo Libretti, Einar Gudlaugsson, Svein R. Kjosavik, Lars Egevad, Emiel A. M. Janssen, Martin Eklund, Kimmo Kartasalo. Artificial intelligence-assisted prostate cancer diagnosis for reduced use of immunohistochemistry. Communications Medicine, 2025, 5(1): 425. https://doi.org/10.1038/s43856-025-01185-yIII. Xiaoyi Ji, Renata Zelic, Oskar Aspegren, Nita Mulliqi, Michelangelo Fiorentino, Francesca Giunchi, Luca Molinaro, Sol Erika Boman, Kelvin Szolnoky, Luana Xuan Liu, Andreas Pettersson, Per Henrik Vincent, Martin Eklund, Olof Akre, Kimmo Kartasalo. Retrospective validation of an artificial intelligence system for diagnostic assessment of prostate biopsies on the ProMort cohort: study protocol. BMJ open, 2025, 15(12): e111361. https://doi.org/10.1136/bmjopen-2025-111361IV. Xiaoyi Ji, Renata Zelic, Oskar Aspegren, Nita Mulliqi, Michelangelo Fiorentino, Francesca Giunchi, Luca Molinaro, Sol Erika Boman, Andreas Pettersson, Lorenzo Richiardi, Per Henrik Vincent, Martin Eklund, Olof Akre, Kimmo Kartasalo. Validation of an AI-based end-to-end model for prostate pathology using long-term archived routine samples. [Manuscript]V. Anders Blilie*, Xiaoyi Ji*, Nita Mulliqi, Richard Salmon, Einar...

8 May 2026
Verified
Preprint
Open access
XJXiaoyi Ji
Xiaoyi Ji
Artificial intelligence (Al) has reached expert-level performance on curated benchmarks for prostate cancer histopathology, yet a gap persists before clinically trustworthy implementation, which requires more than accuracy in model validation. Real-world deployment introduces sources of variation that current development and evaluation designs may have overlooked, including differences across clinical sites, imaging devices, and data characteristics introduced by time. The extent to which contemporary AI systems keep reliable under such conditions, and the mechanisms by which their robustness can be assured, remain to be fully characterized. This thesis aims to investigate the generalization and robustness of AI models for computational prostate cancer pathology across multiple sources of variation and to examine possible strategies for improvement.In Study I, we addressed between-site variation and offered an AI model-agnostic method for color calibration of whole slide image (WSI) scanners. Using a biomaterial-based physical calibrant slide and a chosen color profile reference, cross-scanner generalization of cancer diagnosis and grading performance substantially improved for both a fully supervised convolutional model and a weakly supervised foundation model-based system across multiple cohorts. For example, the concordance of grading with pathologists improved from 0.354 to 0.738 for the internal cohort and from 0.439 to 0.619 for an external cohort with the fully supervised model. The method also outperformed existing computational color normalization approaches, while requiring no additional training data and introducing no visible artifacts.In Study II, we illustrated a secondary clinical application of AI-assisted grading. When a sensitivity-prioritized threshold was picked, an end-to-end attention-based multiple instance learning (ABMIL) model without retraining managed to reduce unnecessary immunohistochemistry orders by 20-44% across three cohorts of diagnostically challenging biopsies without missing any cancer, supporting the broader clinical utility of grading models in streamlining decision-making in prostate pathology and alleviating resource burdens.In Study III, we established a preregistered external validation protocol conforming to STARD-AI, defining data inclusion and exclusion criteria, reference standards, and performance evaluation metrics, while being transparent about the model architecture and data preprocessing pipeline, to enable a rigorous generalization assessment of the AI model on long-term archival material.In Study IV, we executed the protocol from Study III on 10,366 biopsy cores from 1,028 patients archived across 14 Swedish regions between 1998 and 2015. Core-level grading of the model presented comparable performance to several experienced pathologists, with an overall quadratic-weighted kappa of 0.86 against the reference standard. Confusion matrices and survival analysis further supported the model's ability to capture clinically relevant features from tissue morphology. Notably, the model performance remained stable across the 17-year archival span, superior in consistency to the foundation model-based systems.In Study V, we conducted the first investigation of temporal variation within a single scanner on contemporary AI computational pathology models. Evaluating models on WSIs obtained by rescanning a fixed biopsy set every 14 days over one year, we found that foundation model encoders carry strong temporal signatures from their pretraining that task-specific training of the downstream attention aggregator could only partially suppress, even with a training dataset exceeding 60,000 WSIs. In contrast, the end-to-end ABMIL model learned to focus mostly on morphological information and largely ignored irrelevant temporal variation. This representation-level difference was reflected in performance stability, as the end-to-end model maintained consistent self-agreement across the year, whereas several foundation model-based systems showed measurable degradation over the time span.List of scientific papersI. Xiaoyi Ji, Richard Salmon, Nita Mulliqi, Umair Khan, Yinxi Wang, Anders Blilie, Henrik Olsson, Bodil Ginnerup Pedersen, Karina Dalsgaard Sørensen, Benedicte Parm Ulhøi, Svein R. Kjosavik, Emilius A.M. Janssen, Mattias Rantalainen, Lars Egevad, Pekka Ruusuvuori, Martin Eklund, Kimmo Kartasalo. Physical color calibration of digital pathology scanners for robust artificial intelligence-assisted cancer diagnosis. Modern Pathology, 2025, 38(5): 100715. https://doi.org/10.1016/j.modpat.2025.100715II. Anders Blilie*, Nita Mulliqi*, Xiaoyi Ji, Kelvin Szolnoky, Sol Erika Boman, Matteo Titus, Geraldine Martinez Gonzalez, José Asenjo, Marcello Gambacorta, Paolo Libretti, Einar Gudlaugsson, Svein R. Kjosavik, Lars Egevad, Emiel A. M. Janssen, Martin Eklund, Kimmo Kartasalo. Artificial intelligence-assisted prostate cancer diagnosis for reduced use of immunohistochemistry. Communications Medicine, 2025, 5(1): 425. https://doi.org/10.1038/s43856-025-01185-yIII. Xiaoyi Ji, Renata Zelic, Oskar Aspegren, Nita Mulliqi, Michelangelo Fiorentino, Francesca Giunchi, Luca Molinaro, Sol Erika Boman, Kelvin Szolnoky, Luana Xuan Liu, Andreas Pettersson, Per Henrik Vincent, Martin Eklund, Olof Akre, Kimmo Kartasalo. Retrospective validation of an artificial intelligence system for diagnostic assessment of prostate biopsies on the ProMort cohort: study protocol. BMJ open, 2025, 15(12): e111361. https://doi.org/10.1136/bmjopen-2025-111361IV. Xiaoyi Ji, Renata Zelic, Oskar Aspegren, Nita Mulliqi, Michelangelo Fiorentino, Francesca Giunchi, Luca Molinaro, Sol Erika Boman, Andreas Pettersson, Lorenzo Richiardi, Per Henrik Vincent, Martin Eklund, Olof Akre, Kimmo Kartasalo. Validation of an AI-based end-to-end model for prostate pathology using long-term archived routine samples. [Manuscript]V. Anders Blilie*, Xiaoyi Ji*, Nita Mulliqi, Richard Salmon, Einar...