In December 2019, an outbreak of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection occurred in Wuhan, Hubei Province, China and spread across China and beyond. On Febr...

The functional architecture of the human brain has been extensively described in terms of functional connectivity networks, detected from the low–frequency coherent neuronal fluctuations that can be observed in a resting state condition. Little is known, so far, about the changes in functional connectivity and in the topological properties of functional networks, associated with different brain diseases. In this study, we investigated alterations related to mesial temporal lobe epilepsy (mTLE), using resting state functional magnetic resonance imaging on 18 mTLE patients and 27 healthy controls. Functional connectivity among 90 cortical and subcortical regions was measured by temporal correlation. The related values were analyzed to construct a set of undirected graphs. Compared to controls, mTLE patients showed significantly increased connectivity within the medial temporal lobes, but also significantly decreased connectivity within the frontal and parietal lobes, and between frontal and parietal lobes. Our findings demonstrated that a large number of areas in the default-mode network of mTLE patients showed a significantly decreased number of connections to other regions. Furthermore, we observed altered small-world properties in patients, along with smaller degree of connectivity, increased n-to-1 connectivity, smaller absolute clustering coefficients and shorter absolute path length. We suggest that the mTLE alterations observed in functional connectivity and topological properties may be used to define tentative disease markers.

During the diagnostic process, clinicians leverage multimodal information, such as the chief complaint, medical images and laboratory test results. Deep-learning models for aiding diagnosis have yet to meet this requirement of leveraging multimodal information. Here we report a transformer-based representation-learning model as a clinical diagnostic aid that processes multimodal input in a unified manner. Rather than learning modality-specific features, the model leverages embedding layers to convert images and unstructured and structured text into visual tokens and text tokens, and uses bidirectional blocks with intramodal and intermodal attention to learn holistic representations of radiographs, the unstructured chief complaint and clinical history, and structured clinical information such as laboratory test results and patient demographic information. The unified model outperformed an image-only model and non-unified multimodal diagnosis models in the identification of pulmonary disease (by 12% and 9%, respectively) and in the prediction of adverse clinical outcomes in patients with COVID-19 (by 29% and 7%, respectively). Unified multimodal transformer-based models may help streamline the triaging of patients and facilitate the clinical decision-making process.