In recent years, applying multi-modal large language models (MLLMs) in various fields has achieved remarkable success. However, as the foundation model for many downstream tasks, MLLMs comprise the well-known Transformer network, which has a less efficient quadratic computation complexity. In this study, we introduce Cobra, a multi-modal large-scale language model built upon a state-space model, which has demonstrated significant potential in efficiently handling long sequences with fast inference and linear scalability concerning sequence length. Specifically, Cobra involves replacing Transformer-based backbone models (e.g., LLaMA or Phi) with pre-trained Mamba language models. We then empirically explore effective strategies for aligning visual and textual modalities and integrating various pre-trained Mamba model variants with visual encoders. Experiments across various multi-modal benchmarks demonstrate that: (i) Cobra performs 3× ∼ 4× faster than the most computationally efficient state-of-the-art methods, e.g., LLaVA-Phi and MobileVLM v2. Additionally, its performance is significantly enhanced thanks to the implementation of linear sequential modeling. (ii) Cobra fine-tunes a small parameter (∼48% of model parameters), leading to a significant improvement in overall performance compared to LLaVA.

Graph Neural Networks (GNNs) are state-of-the-art models for performing prediction tasks on graphs. While existing GNNs have shown great performance on various tasks related to graphs, little attention has been paid to the scenario where out-of-distribution (OOD) nodes exist in the graph during training and inference. Borrowing the concept from CV and NLP, we define OOD nodes as nodes with labels unseen from the training set. Since a lot of networks are automatically constructed by programs, real-world graphs are often noisy and may contain nodes from unknown distributions. In this work, we define the problem of graph learning with out-of-distribution nodes. Specifically, we aim to accomplish two tasks: 1) detect nodes which do not belong to the known distribution and 2) classify the remaining nodes to be one of the known classes. We demonstrate that the connection patterns in graphs are informative for outlier detection, and propose Out-of-Distribution Graph Attention Network (OODGAT), a novel GNN model which explicitly models the interaction between different kinds of nodes and separate inliers from outliers during feature propagation. Extensive experiments show that OODGAT outperforms existing outlier detection methods by a large margin, while being better or comparable in terms of in-distribution classification.

Vision-and-language navigation (VLN) is a challenging task that requires an agent to navigate in real-world environments by understanding natural language instructions and visual information received in real time. Prior works have implemented VLN tasks on continuous environments or physical robots, all of which use a fixed-camera configuration due to the limitations of datasets, such as 1.5-m height, 90° horizontal field of view (HFOV), and so on. However, real-life robots with different purposes have multiple camera configurations, and the huge gap in visual information makes it difficult to directly transfer the learned navigation skills between various robots. In this brief, we propose a visual perception generalization strategy based on meta-learning, which enables the agent to fast adapt to a new camera configuration. In the training phase, we first locate the generalization problem to the visual perception module and then compare two meta-learning algorithms for better generalization in seen and unseen environments. One of them uses the model-agnostic meta-learning (MAML) algorithm that requires few-shot adaptation, and the other refers to a metric-based meta-learning method with a feature-wise affine transformation (AT) layer. The experimental results on the VLN-CE dataset demonstrate that our strategy successfully adapts the learned navigation skills to new camera configurations, and the two algorithms show their advantages in seen and unseen environments respectively.

Tau protein, a microtubule-associated protein, has a high specific expression in neurons and axons. Because traumatic spinal cord injury mainly affects neurons and axons, we speculated that tau protein may be a promising biomarker to reflect the degree of spinal cord injury and prognosis of motor function. In this study, 160 female Sprague-Dawley rats were randomly divided into a sham group, and mild, moderate, and severe spinal cord injury groups. A laminectomy was performed at the T8 level to expose the spinal cord in all groups. A contusion lesion was made with the NYU-MASCIS impactor by dropping a 10 g rod from heights of 12.5 mm (mild), 25 mm (moderate) and 50 mm (severe) upon the exposed dorsal surface of the spinal cord. Tau protein levels were measured in serum and cerebrospinal fluid samples at 1, 6, 12, 24 hours, 3, 7, 14 and 28 days after operation. Locomotor function of all rats was assessed using the Basso, Beattie and Bresnahan locomotor rating scale. Tau protein concentration in the three spinal cord injury groups (both in serum and cerebrospinal fluid) rapidly increased and peaked at 12 hours after spinal cord injury. Statistically significant positive linear correlations were found between tau protein level and spinal cord injury severity in the three spinal cord injury groups, and between the tau protein level and Basso, Beattie, and Bresnahan locomotor rating scale scores. The tau protein level at 12 hours in the three spinal cord injury groups was negatively correlated with Basso, Beattie, and Bresnahan locomotor rating scale scores at 28 days (serum: r = −0.94; cerebrospinal fluid: r = −0.95). Our data suggest that tau protein levels in serum and cerebrospinal fluid might be a promising biomarker for predicting the severity and functional outcome of traumatic spinal cord injury.