Radiomics investigates the predictive role of quantitative parameters calculated from radiological images. In oncology, tumour segmentation constitutes a crucial step of the radiomic workflow. Manual segmentation is time-consuming and prone to inter-observer variability. In this study, a state-of-the-art deep-learning network for automatic segmentation (nnU-Net) was applied to computed tomography images of lung tumour patients, and its impact on the performance of survival radiomic models was assessed. In total, 899 patients were included, from two proprietary and one public datasets. Different network architectures (2D, 3D) were trained and tested on different combinations of the datasets. Automatic segmentations were compared to reference manual segmentations performed by physicians using the DICE similarity coefficient. Subsequently, the accuracy of radiomic models for survival classification based on either manual or automatic segmentations were compared, considering both hand-crafted and deep-learning features. The best agreement between automatic and manual contours (DICE = 0.78 ± 0.12) was achieved averaging 2D and 3D predictions and applying customised post-processing. The accuracy of the survival classifier (ranging between 0.65 and 0.78) was not statistically different when using manual versus automatic contours, both with hand-crafted and deep features. These results support the promising role nnU-Net can play in automatic segmentation, accelerating the radiomic workflow without impairing the models’ accuracy. Further investigations on different clinical endpoints and populations are encouraged to confirm and generalise these findings.

Decoding visual representations from human brain activity has emerged as a thriving research domain, particularly in the context of brain-computer interfaces. Our study presents an innovative method that employs knowledge distillation to train an EEG classifier and reconstruct images from the ImageNet and THINGS-EEG 2 datasets using only electroencephalography (EEG) data from participants who have viewed the images themselves (i.e. "brain decoding"). We analyzed EEG recordings from 6 participants for the ImageNet dataset and 10 for the THINGS-EEG 2 dataset, exposed to images spanning unique semantic categories. These EEG readings were converted into spectrograms, which were then used to train a convolutional neural network (CNN), integrated with a knowledge distillation procedure based on a pre-trained Contrastive Language-Image Pre-Training (CLIP)-based image classification teacher network. This strategy allowed our model to attain a top-5 accuracy of 87%, significantly outperforming a standard CNN and various RNN-based benchmarks. Additionally, we incorporated an image reconstruction mechanism based on pre-trained latent diffusion models, which allowed us to generate an estimate of the images that had elicited EEG activity. Therefore, our architecture not only decodes images from neural activity but also offers a credible image reconstruction from EEG only, paving the way for, e.g., swift, individualized feedback experiments.

To-date, brain decoding literature has focused on single-subject studies, that is, reconstructing stimuli presented to a subject under fMRI acquisition from the fMRI activity of the same subject. The objective of this study is to introduce a generalization technique that enables the decoding of a subject’s brain based on fMRI activity of another subject, that is, cross-subject brain decoding. To this end, we also explore cross-subject data alignment techniques. Data alignment is the attempt to register different subjects in a common anatomical or functional space for further and more general analysis. We utilized the Natural Scenes Dataset, a comprehensive 7T fMRI experiment focused on vision of natural images. The dataset contains fMRI data from multiple subjects exposed to 9,841 images, where 982 images have been viewed by all subjects. Our method involved training a decoding model on one subject’s data, aligning new data from other subjects to this space, and testing the decoding on the second subject based on information aligned to the first subject. We also compared different techniques for fMRI data alignment, specifically ridge regression, hyper alignment, and anatomical alignment. We found that cross-subject brain decoding is possible, even with a small subset of the dataset, specifically, using the common data, which are around 10% of the total data, namely 982 images, with performances in decoding comparable to the ones achieved by single-subject decoding. Cross-subject decoding is still feasible using half or a quarter of this number of images with slightly lower performances. Ridge regression emerged as the best method for functional alignment in fine-grained information decoding, outperforming all other techniques. By aligning multiple subjects, we achieved high-quality brain decoding and a potential reduction in scan time by 90%. This substantial decrease in scan time could open up unprecedented opportunities for more efficient experiment execution and further advancements in the field, which commonly requires prohibitive (20 hours) scan time per subject.

This study delves into the crucial aspect of network topology in artificial neural networks (NNs) and its impact on model performance. Addressing the need to comprehend how network structures influence learning capabilities, the research contrasts traditional multilayer perceptrons (MLPs) with models built on various complex topologies using novel network generation techniques. Drawing insights from synthetic datasets, the study reveals the remarkable accuracy of complex NNs, particularly in high-difficulty scenarios, outperforming MLPs. Our exploration extends to real-world datasets, highlighting the task-specific nature of optimal network topologies and unveiling trade-offs, including increased computational demands and reduced robustness to graph damage in complex NNs compared to MLPs. This research underscores the pivotal role of complex topologies in addressing challenging learning tasks. However, it also signals the necessity for deeper insights into the complex interplay among topological attributes influencing NN performance. By shedding light on the advantages and limitations of complex topologies, this study provides valuable guidance for practitioners and paves the way for future endeavors to design more efficient and adaptable neural architectures across various applications.

We propose a novel approach for the non-invasive quantification of dynamic PET imaging data, focusing on the arterial input function (AIF) without the need for invasive arterial cannulation. Our method utilizes a combination of three-dimensional depth-wise separable convolutional layers and a physically informed deep neural network to incorporatea priori knowledge about the AIF's functional form and shape, enabling precise predictions of the concentrations of [¹¹C]PBR28 in whole blood and the free tracer in metabolite-corrected plasma. We found a robust linear correlation between our model's predicted AIF curves and those obtained through traditional, invasive measurements. We achieved an average cross-validated Pearson correlation of 0.86 for whole blood and 0.89 for parent plasma curves. Moreover, our method's ability to estimate the volumes of distribution across several key brain regions - without significant differences between the use of predicted versus actual AIFs in a two-tissue compartmental model - successfully captures the intrinsic variability related to sex, the binding affinity of the translocator protein (18 kDa), and age. These results not only validate our method's accuracy and reliability but also establish a foundation for a streamlined, non-invasive approach to dynamic PET data quantification. By offering a precise and less invasive alternative to traditional quantification methods, our technique holds significant promise for expanding the applicability of PET imaging across a wider range of tracers, thereby enhancing its utility in both clinical research and diagnostic settings.

Objective. Brain decoding is a field of computational neuroscience that aims to infer mental states or internal representations of perceptual inputs from measurable brain activity. This study proposes a novel approach to brain decoding that relies on semantic and contextual similarity. Approach. We use several functional magnetic resonance imaging (fMRI) datasets of natural images as stimuli and create a deep learning decoding pipeline inspired by the bottom-up and top-down processes in human vision. Our pipeline includes a linear brain-to-feature model that maps fMRI activity to semantic visual stimuli features. We assume that the brain projects visual information onto a space that is homeomorphic to the latent space of last layer of a pretrained neural network, which summarizes and highlights similarities and differences between concepts. These features are categorized in the latent space using a nearest-neighbor strategy, and the results are used to retrieve images or condition a generative latent diffusion model to create novel images. Main results. We demonstrate semantic classification and image retrieval on three different fMRI datasets: Generic Object Decoding (vision perception and imagination), BOLD5000, and NSD. In all cases, a simple mapping between fMRI and a deep semantic representation of the visual stimulus resulted in meaningful classification and retrieved or generated images. We assessed quality using quantitative metrics and a human evaluation experiment that reproduces the multiplicity of conscious and unconscious criteria that humans use to evaluate image similarity. Our method achieved correct evaluation in over 80% of the test set. Significance. Our study proposes a novel approach to brain decoding that relies on semantic and contextual similarity. The results demonstrate that measurable neural correlates can be linearly mapped onto the latent space of a neural network to synthesize images that match the original content. These findings have implications for both cognitive neuroscience and artificial intelligence.