When data is pooled across multiple sites, the extracted features are confounded by site effects. Harmonization methods attempt to correct these site effects while preserving the biological variability within the features. However, little is known about the sample size requirement for effectively learning the harmonization parameters and their relationship with the increasing number of sites. In this study, we performed experiments to find the minimum sample size required to achieve multisite harmonization (using neuroHarmonize) using volumetric and surface features by leveraging the concept of learning curves. Our first two experiments show that site-effects are effectively removed in a univariate and multivariate manner; however, it is essential to regress the effect of covariates from the harmonized data additionally. Our following two experiments with actual and simulated data showed that the minimum sample size required for achieving harmonization grows with the increasing average Mahalanobis distances between the sites and their reference distribution. We conclude by positing a general framework to understand the site effects using the Mahalanobis distance. Further, we provide insights on the various factors in a cross-validation design to achieve optimal inter-site harmonization.
T1-weighted (T1w) MRI is widely used in clinical neuroimaging for studying brain structure and its changes, including those related to neurodegenerative diseases, and as anatomical reference for analysing other modalities. Ensuring high-quality T1w scans is vital as image quality affects reliability of outcome measures. However, visual inspection can be subjective and time consuming, especially with large datasets. The effectiveness of automated quality control (QC) tools for clinical cohorts remains uncertain. In this study, we used T1w scans from elderly participants within ageing and clinical populations to test the accuracy of existing QC tools with respect to visual QC and to establish a new quality prediction framework for clinical research use. Four datasets acquired from multiple scanners and sites were used (N = 2438, 11 sites, 39 scanner manufacturer models, 3 field strengths—1.5T, 3T, 2.9T, patients and controls, average age 71 ± 8 years). All structural T1w scans were processed with two standard automated QC pipelines (MRIQC and CAT12). The agreement of the accept–reject ratings was compared between the automated pipelines and with visual QC. We then designed a quality prediction framework that combines the QC measures from the existing automated tools and is trained on clinical research datasets. We tested the classifier performance using cross-validation on data from all sites together, also examining the performance across diagnostic groups. We then tested the generalisability of our approach when leaving one site out and explored how well our approach generalises to data from a different scanner manufacturer and/or field strength from those used for training, as well as on an unseen new dataset of healthy young participants with movement-related artefacts. Our results show significant agreement between automated QC tools and visual QC (Kappa = 0.30 with MRIQC predictions; Kappa = 0.28 with CAT12’s rating) when considering the entire dataset, but the agreement was highly variable across datasets. Our proposed robust undersampling boost (RUS) classifier achieved 87.7% balanced accuracy on the test data combined from different sites (with 86.6% and 88.3% balanced accuracy on scans from patients and controls, respectively). This classifier was also found to be generalisable on different combinations of training and test datasets (average balanced accuracy of leave-one-site-out = 78.2%; exploratory models on field strengths and manufacturers = 77.7%; movement-related artefact dataset when including 1% scans in the training = 88.5%). While existing QC tools may not be robustly applicable to datasets comprising older adults, they produce quality metrics that can be leveraged to train more robust quality control classifiers for ageing and clinical cohorts.
7 May 2021 in International Journal of Methods in Psychiatric Research
Objective: The Accelerator program for Discovery in Brain disorders using Stem cells (ADBS) is a longitudinal study on five cohorts of patients with major psychiatric disorders from genetically high‐risk families, their unaffected first‐degree relatives, and healthy subjects. We describe the ADBS protocols for acquisition, quality assurance (QA), and quality check (QC) for multimodal magnetic resonance brain imaging studies.Methods: We describe the acquisition and QC protocols for structural, functional, and diffusion images. For QA, we acquire proton density and functional images on phantoms, along with repeated scans on human volunteer. We describe the analysis of phantom data and test–retest reliability of volumetric and diffusion measures.Results: Analysis of acquired phantom data shows linearity of proton density signal with increasing proton fraction, and an overall stability of various spatial and temporal QA measures. Examination of dice coefficient and statistical analyses of coefficient of variation in test–retest data on the human volunteer showed consistency of volumetric and diffusivity measures at whole‐brain, regional, and voxel‐level.Conclusion: The described acquisition and QA–QC procedures can yield consistent and reliable quantitative measures. It is expected that this longitudinal neuroimaging dataset will, upon its release, serve the scientific community well and pave the way for interesting discoveries.
Introduction: T1-weighted MRI is widely used in clinical neuroimaging for studying brain structure and its changes, including those related to neurodegenerative diseases, and as anatomical reference for analysing other modalities. Ensuring high-quality T1-weighted scans is vital as image quality affects reliability of outcome measures. However, visual inspection can be subjective and time-consuming, especially with large datasets. The effectiveness of automated quality control (QC) tools for clinical cohorts remains uncertain. In this study, we used T1w scans from elderly participants within ageing and clinical populations to test the accuracy of existing QC tools with respect to visual QC and to establish a new quality prediction framework for clinical research use. Methods: Four datasets acquired from multiple scanners and sites were used ( N = 2438, 11 sites, 39 scanner manufacturer models, 3 field strengths – 1.5T, 3T, 2.9T, patients and controls, average age 71 ± 8 years). All structural T1w scans were processed with two standard automated QC pipelines (MRIQC and CAT12). The agreement of the accept-reject ratings was compared between the automated pipelines and with visual QC. We then designed a quality prediction framework that combines the QC measures from the existing automated tools and is trained on clinical datasets. We tested the classifier performance using cross-validation on data from all sites together, also examining the performance across diagnostic groups. We then tested the generalisability of our approach when leaving one site out and explored how well our approach generalises to data from a different scanner manufacturer and/or field strength from those used for training. Results: Our results show significant agreement between automated QC tools and visual QC (Kappa=0.30 with MRIQC predictions; Kappa=0.28 with CAT12’s rating) when considering the entire dataset, but the agreement was highly variable across datasets. Our proposed robust undersampling boost (RUS) classifier achieved 87.7% balanced accuracy on the test data combined from different sites (with 86.6% and 88.3% balanced accuracy on scans from patients and controls respectively). This classifier was also found to be generalisable on different combinations of training and test datasets (leave-one-site-out = 78.2% average balanced accuracy; exploratory models = 77.7% average balanced accuracy). Conclusion: While existing QC tools may not be robustly applicable to datasets comprised of older adults who have a higher rate of atrophy, they produce quality metrics that can be leveraged to train a more robust quality control classifiers for ageing and clinical cohorts.