It has been recently shown that deep learning models such as convolutional neural networks (CNN), deep belief networks (DBN) and recurrent neural networks (RNN), exhibited remarkable ability in modeling and representing fMRI data for the understanding of functional activities and networks because of their superior data representation capability and wide availability of effective deep learning tools. For example, spatial and/or temporal patterns of functional brain activities embedded in fMRI data can be effectively characterized and modeled by a variety of CNN/DBN/RNN deep learning models as shown in recent studies. However, it has been rarely investigated whether it is possible to directly infer hierarchical brain networks from volumetric fMRI data using deep learning models such as DBN. The perceived difficulties of such studies include very large number of input variables, very large number of training parameters, the lack of effective software tools, the challenge of results interpretation, and etc. To bridge these technical gaps, we designed a novel volumetric sparse deep belief network (VS-DBN) model and implemented it through the popular TensorFlow open source platform to reconstruct hierarchical brain networks from volumetric fMRI data based on the Human Connectome Project (HCP) 900 subjects release. Our experimental results showed that a large number of interpretable and meaningful brain networks can be robustly reconstructed from HCP 900 subjects in a hierarchical fashion, and importantly, these brain networks exhibit reasonably good consistency and correspondence across multiple HCP task-based fMRI datasets. Our work contributed a new general deep learning framework for inferring multiscale volumetric brain networks and offered novel insights into the hierarchical organization of functional brain architecture.

Objective . Recently, deep learning models have been successfully applied in functional magnetic resonance imaging (fMRI) modeling and associated applications. However, there still exist at least two challenges. Firstly, due to the lack of sufficient data, deep learning models tend to suffer from overfitting in the training process. Secondly, it is still challenging to model the temporal dynamics from fMRI, due to that the brain state is continuously changing over scan time. In addition, existing methods rarely studied and applied fMRI data augmentation. Approach . In this work, we construct a deep recurrent variational auto-encoder (DRVAE) that combined variational auto-encoder and recurrent neural network, aiming to address all of the above mentioned challenges. The encoder of DRVAE can extract more generalized temporal features from assumed Gaussian distribution of input data, and the decoder of DRVAE can generate new data to increase training samples and thus partially relieve the overfitting issue. The recurrent layers in DRVAE are designed to effectively model the temporal dynamics of functional brain activities. LASSO (least absolute shrinkage and selection operator) regression is applied on the temporal features and input fMRI data to estimate the corresponding spatial networks. Main results . Extensive experimental results on seven tasks from HCP dataset showed that the DRVAE and LASSO framework can learn meaningful temporal patterns and spatial networks from both real data and generated data. The results on group-wise data and single subject suggest that the brain activities may follow certain distribution. Moreover, we applied DRVAE on four resting state fMRI datasets from ADHD-200 for data augmentation, and the results showed that the classification performances on augmented datasets have been considerably improved. Significance . The proposed method can not only derive meaningful temporal features and spatial networks from fMRI, but also generate high-quality new data for fMRI data augmentation and associated applications.

In the neuroimaging and brain mapping communities, researchers have proposed a variety of computational methods to map functional brain networks (FBNs). Recently, it has been proven that deep learning (DL) can be applied on functional magnetic resonance image (fMRI) data with superb representation power over the traditional machine learning methods. However, due to the lack of labeled data and the high dimension of fMRI volume images, DL suffers from overfitting in both supervised and unsupervised training processes. In this work, we proposed a novel generative model: deep variational autoencoder (DVAE) to tackle the challenge of insufficient data and incomplete supervision. The experimental results showed that the representations learned by DVAE are interpretable and meaningful compared to those learned from well-known sparse dictionary learning (SDL). Besides, the organization of some FBN patterns derived from different layers in DVAE was observed in a hierarchical fashion. Furthermore, we showed that DVAE has better performance on small dataset over autoencoder (AE). By using attention deficit hyperactivity disorder (ADHD)-200 dataset as our test bed, we constructed a DVAE-based modeling and classification pipeline in which all subjects’ functional connectivities estimated by FBNs were taken as input features to train a classifier. Finally, the results achieved by our pipeline reached state-of-the-art classification accuracies on three ADHD-200 sites compared with other fMRI-based methods.

One of most primitive problems by unmanned underwater vehicle intelligent swarm (UIS) is coordination control, which has a great significance for realization of target hunting with great performance of efficiency and robustness. Existing studies concentrate on behavior-based centralized or distributed control approaches with the prior knowledge and mostly do not elaborately consider behavior conflicts and constraint differences. Therefore, a novel behavior-driven coordination control framework including topology architecture and swarm control which is inspired by immune mechanism, is investigated for target hunting of heterogeneous UIS under unknown and uncertain environment in this paper. For topology architecture, a hybrid non-central distributed topology is developed as a novel immune-inspired architecture to regulate agents with self-organizational and fault-tolerance features. For swarm control, a dual-layer switching control scheme composed by global control and local control, is proposed to drive behaviors via behavioral-intensity, the trigger of switching is when the target is detected. The global control approach is employed to search target, in which two constraints of energy consumption and healthy-state are considered to achieve good operational reliability. While the local control approach is developed to form the dynamic alliance of tracking and capturing, in which behavioral-intensity control strategy for behavior aggregation and decision-making control strategy for behavior selection are respectively designed to avoid behavior conflicts. Simulation results demonstrate that proposed framework can accomplish hunting under various situations such as hunter agent is random or fixed distribution, and the number of targets asynchronously appears. It is confirmed that our framework is capable of achieving the target hunting under unknown and uncertain environment with greater efficiency and robustness.

Electroencephalogram (EEG) is one of the most widely used brain computer interface (BCI) approaches. Despite the success of existing EEG approaches in brain state recognition studies, it is still challenging to differentiate brain states via explainable and generalizable deep learning approaches. In other words, how to explore meaningful and distinguishing features and how to overcome the huge variability and overfitting problem still need to be further studied. To alleviate these challenges, in this work, a multiple random fragment search-based multilayer recurrent neural network (MRFS-MRNN) is proposed to improve the differentiating performance and explore meaningful patterns. Specifically, an explainable MRNN module is proposed to capture the temporal dependences preserved in EEG time series. Besides, a MRFS module is designed to cut multiple random fragments from the entire EEG signal time course to improve the effectiveness of brain state differentiating ability. MRFS-MRNN is concatenatedto effectively overcome the huge variabilities and overfitting problems. Experiment results demonstrate that the proposed MRFS-MRNN model not only has excellent differentiating performance, but also has good explanation and generalization ability. The classification accuracies reach as high as 95.18% for binary classification and 89.19% for four-category classification on the individual level. Similarly, 95.53% and 85.84% classification accuracies are obtained for the binary and four-category classification on the group level. What’s more, 94.28% and 85.43% classification accuracies of binary and four-category classifications are achieved for predicting brand new subjects. The experiment results showed that the proposed method outperformed other state-of-the-art (SOTA) models on the same underlying data and improved the explanation and generalization ability.

Accurate genotyping of the epidermal growth factor receptor (EGFR) is critical for the treatment planning of lung adenocarcinoma. Currently, clinical identification of EGFR genotyping highly relies on biopsy and sequence testing which is invasive and complicated. Recent advancements in the integration of computed tomography (CT) imagery with deep learning techniques have yielded a non-invasive and straightforward way for identifying EGFR profiles. However, there are still many limitations for further exploration: 1) most of these methods still require physicians to annotate tumor boundaries, which are time-consuming and prone to subjective errors; 2) most of the existing methods are simply borrowed from computer vision field which does not sufficiently exploit the multi-level features for final prediction. To solve these problems, we propose a Denseformer framework to identify EGFR mutation status in a real end-to-end fashion directly from 3D lung CT images. Specifically, we take the 3D whole-lung CT images as the input of the neural network model without manually labeling the lung nodules. This is inspired by the medical report that the mutational status of EGFR is associated not only with the local tumor nodules but also with the microenvironment surrounded by the whole lung. Besides, we design a novel Denseformer network to fully explore the distinctive information across the different level features. The Denseformer is a novel network architecture that combines the advantages of both convolutional neural network (CNN) and Transformer. Denseformer directly learns from the 3D whole-lung CT images, which preserves the spatial location information in the CT images. To further improve the model performance, we designed a combined Transformer module. This module employs the Transformer Encoder to globally integrate the information of different levels and layers and use them as the basis for the final prediction. The proposed model has been tested on a lung adenocarcinoma dataset collected at the Affiliated Hospital of Zunyi Medical University. Extensive experiments demonstrated the proposed method can effectively extract meaningful features from 3D CT images to make accurate predictions. Compared with other state-of-the-art methods, Denseformer achieves the best performance among current methods using deep learning to predict EGFR mutation status based on a single modality of CT images.