Object recognition in real-world environments is one of the fundamental and key tasks in computer vision and robotics communities. With the advanced sensing technologies and low-cost depth sensors, the high-quality RGB and depth images can be recorded synchronously, and the object recognition performance can be improved by jointly exploiting them. RGB-D-based object recognition has evolved from early methods that using hand-crafted representations to the current state-of-the-art deep learning-based methods. With the undeniable success of deep learning, especially convolutional neural networks (CNNs) in the visual domain, the natural progression of deep learning research points to problems involving larger and more complex multimodal data. In this paper, we provide a comprehensive survey of recent multimodal CNNs (MMCNNs)-based approaches that have demonstrated significant improvements over previous methods. We highlight two key issues, namely, training data deficiency and multimodal fusion. In addition, we summarize and discuss the publicly available RGB-D object recognition datasets and present a comparative performance evaluation of the proposed methods on these benchmark datasets. Finally, we identify promising avenues of research in this rapidly evolving field. This survey will not only enable researchers to get a good overview of the state-of-the-art methods for RGB-D-based object recognition but also provide a reference for other multimodal machine learning applications, e.g., multimodal medical image fusion, audio-visual speech recognition, and multimedia retrieval and generation.

Affected by the downstream load of the line, the low-voltage AC series arc fault signal presents nonlinear, non-stationary and random characteristics, which bring great difficulties to the extraction of arc fault features and lead to low arc detection accuracy. To solve this problem, a low-voltage AC series arc fault detection method based on strong discriminative features and sensitive components is proposed. In this work, firstly, the arc fault current signal is collected based on the self-built experimental platform, and the arc fault current components are obtained by the complete ensemble empirical mode decomposition with the adaptive noise (CEEMDAN) method. Secondly, 16 feature indexes based on time window are used to describe the component signals, and an effective feature selection method based on the maximum mutual information coefficient and the significance of feature changes is proposed to realize the mining of highly identifiable features. Then, an improved hierarchical clustering algorithm and a sensitive component selection strategy combining the saliency of feature changes are proposed to eliminate the interference of redundant components. Finally, a strong discriminative feature library of sensitive components of current signal is constructed, and series arc fault detection is realized based on support vector machine. Experimental results show that the proposed method is feasible and effective in arc current feature extraction and fault detection, and provides a reference for low-voltage AC series arc fault detection.

Super-resolution (SR) technology plays a crucial role in improving the spatial resolution of remote sensing images so as to overcome the physical limitations of spaceborne imaging systems. Although deep convolutional neural networks have achieved promising results, most of them overlook the advantage of self-similarity information across different scales and high-dimensional features after the upsampling layers. To address the problem, we propose a hybrid-scale hierarchical transformer network (HSTNet) to achieve faithful remote sensing image SR. Specifically, we propose a hybrid-scale feature exploitation module to leverage the internal recursive information in single and cross scales within the images. To fully leverage the high-dimensional features and enhance discrimination, we designed a cross-scale enhancement transformer to capture long-range dependencies and efficiently calculate the relevance between high-dimension and low-dimension features. The proposed HSTNet achieves the best result in PSNR and SSIM with the UCMecred dataset and AID dataset. Comparative experiments demonstrate the effectiveness of the proposed methods and prove that the HSTNet outperforms the state-of-the-art competitors both in quantitative and qualitative evaluations.

With the rapid development of electronic payment technologies, facial recognition-based payment systems have become increasingly popular and indispensable. However, the majority of facial recognition payment systems are vulnerable to being manipulated by facial deepfake technology, and it would be a serious threat to personal property and privacy. In order to effectively defend deepfake models on the premise of minimizing alterations to the original image, we propose a union-saliency attack model which is a well-trained deepfake model while maintaining plausible detail of the original face images. To this aim, we derive a union mask mechanism to accurately determine facial region as a prior in guiding the subsequent perturbations, with the objective of minimizing the information loss on input images. Additionally, we propose a novel structural similarity loss and a noise generator to minimize detail degradation. Experiments prove that the proposed method can interfere with deepfake models effectively and minimize the distortion of the original image simultaneously.

Object counting, defined as the task of accurately predicting the number of objects in static images or videos, has recently attracted considerable interest. However, the unavoidable presence of background noise prevents counting performance from advancing further. To address this issue, we created a group and graph attention network (GGANet) for dense object counting. GGANet is an encoder–decoder architecture incorporating a group channel attention (GCA) module and a learnable graph attention (LGA) module. The GCA module groups the feature map into several subfeatures, each of which is assigned an attention factor through the identical channel attention. The LGA module views the feature map as a graph structure in which the different channels represent diverse feature vertices, and the responses between channels represent edges. The GCA and LGA modules jointly avoid the interference of irrelevant pixels and suppress the background noise. Experiments are conducted on four crowd-counting datasets, two vehicle-counting datasets, one remote-sensing counting dataset, and one few-shot object-counting dataset. Comparative results prove that the proposed abbr achieves superior counting performance.