Object tracking in RGB-thermal (RGB-T) videos is increasingly used in many fields due to the all-weather and all-day working capability of the dual-modality imaging system, as well as the rapid development of low-cost and miniaturized infrared camera technology. However, it is still very challenging to effectively fuse dual-modality information to build a robust RGB-T tracker. In this paper, an RGB-T object tracking algorithm based on a modal-aware attention network and competitive learning (MaCNet) is proposed, which includes a feature extraction network, modal-aware attention network, and classification network. The feature extraction network adopts the form of a two-stream network to extract features from each modality image. The modal-aware attention network integrates the original data, establishes an attention model that characterizes the importance of different feature layers, and then guides the feature fusion to enhance the information interaction between modalities. The classification network constructs a modality-egoistic loss function through three parallel binary classifiers acting on the RGB branch, the thermal infrared branch, and the fusion branch, respectively. Guided by the training strategy of competitive learning, the entire network is fine-tuned in the direction of the optimal fusion of the dual modalities. Extensive experiments on several publicly available RGB-T datasets show that our tracker has superior performance compared to other latest RGB-T and RGB tracking approaches.
Due to the problem of a small amount of EEG samples and relatively high dimensionality of electroencephalogram (EEG) features, feature selection plays an essential role in EEG-based emotion recognition. However, current EEG-based emotion recognition studies utilize a problem transformation approach to transform multi-dimension emotional labels into single-dimension labels, and then implement commonly used single-label feature selection methods to search feature subsets, which ignores the relations between different emotional dimensions. To tackle the problem, we propose an efficient EEG feature selection method for multi-dimension emotion recognition (EFSMDER) via local and global label relevance. First, to capture the local label correlations, EFSMDER implements orthogonal regression to map the original EEG feature space into a low-dimension space. Then, it employs the global label correlations in the original multi-dimension emotional label space to effectively construct the label information in the low-dimension space. With the aid of local and global relevance information, EFSMDER can conduct representational EEG feature subset selection. Three EEG emotional databases with multi-dimension emotional labels were used for performance comparison between EFSMDER and fourteen state-of-the-art methods, and the EFSMDER method achieves the best multi-dimension classification accuracies of 86.43, 84.80, and 97.86 percent on the DREAMER, DEAP, and HDED datasets, respectively.
10 September 2024 in IEEE Internet of Things Journal
The 6G network is expected to accommodate a wide array of connected devices, supporting diverse services from any location at any time. In this article, we introduce an aerial mobile edge computing (MEC) framework composed of high-altitude platforms (HAPs) and low-altitude unmanned aerial vehicles (UAVs), to cater to computing offloading for Internet of Things (IoT) devices, particularly in rural/remote areas or disaster zones. The framework accommodates various types of tasks, each computed by the corresponding Docker container. The objective is to achieve optimal workload fairness for UAVs while simultaneously minimizing the weighted processing costs among IoT devices in terms of task computation latency and energy consumption over the long term. This is achieved by jointly optimizing the flight trajectories and Docker image caching decisions of the UAVs with limited storage capacities, alongside ensuring service fairness for IoT devices. We tailor a multiagent deep deterministic policy gradient (MADDPG)-based approach to solve the long-term joint optimization problem, normalizing continuous actions and sampling discrete actions by generalizing the Gumbel-Softmax reparameterization trick. Experimental results indicate that our approach significantly outperforms benchmark schemes in terms of processing delay, energy consumption, and fairness.
Visual object tracking in unmanned aerial vehicle (UAV) videos plays an important role in a variety of fields, such as traffic data collection, traffic monitoring, as well as film and television shooting. However, it is still challenging to track the target robustly in UAV vision task due to several factors such as appearance variation, background clutter, and severe occlusion. In this paper, we propose a novel two-stage UAV tracking framework, which includes a target detection stage based on multifeature discrimination and a bounding-box estimation stage based on the instance-aware attention network. In the target detection stage, we explore a feature representation scheme for a small target that integrates handcrafted features, low-level deep features, and high-level deep features. Then, the correlation filter is used to roughly predict target location. In the bounding-box estimation stage, an instance-aware intersection over union (IoU)-Net is integrated together with an instance-aware attention network to estimate the target size based on the bounding-box proposals generated in the target detection stage. Extensive experimental results on the UAV123 and UAVDT datasets show that our tracker, running at over 25 frames per second (FPS), has superior performance as compared with state-of-the-art UAV visual tracking approaches.
28 February 2025 in IEEE Transactions on Geoscience and Remote Sensing
Mamba has gained significant attention for its outstanding long-range context modeling capability while maintaining linear complexity, compared with Transformer. In this paper, a hybrid Mamba and Convolutional Neural Network (CNN) architecture is proposed for Remote Sensing Image Change Detection (RSICD), named Hybrid-MambaCD, which leverages the advantages of CNN for local detail information extraction and Mamba for global context information extraction, providing an efficient solution for RSICD tasks. Firstly, Region-Channel Attention Mechanism (RCAM) is designed to enhance the CNN features from both channel and region dimensions, enabling the network to focus more on change regions while suppressing interference from background areas. Secondly, an Iterative Global-local Feature Fusion (IGLFF) strategy is proposed, which performs adaptive weighted fusion of global and local features across multiple scales in a progressive manner, enhancing the representation ability of the features. Experimental results on three public datasets of LEVIR-CD, WHU-CD, and DSIFN-CD show that, compared to the existing RSICD methods, the proposed Hybrid-MambaCD achieves the state-of-the-arts detection performance.