RGB salient object detection (SOD) has made great progress. However, the performance of this single-modal salient object detection will be significantly decreased when encountering some challenging scenes, such as low light or darkness. To deal with the above challenges, thermal infrared (T) image is introduced into the salient object detection. This fused modal is called RGB-T salient object detection. To achieve deep mining of the unique characteristics of single modal and the full integration of cross-modality information, a novel Cross-Guided Fusion Network (CGFNet) for RGB-T salient object detection is proposed. Specifically, a Cross-Scale Alternate Guiding Fusion (CSAGF) module is proposed to mine the high-level semantic information and provide global context support. Subsequently, we design a Guidance Fusion Module (GFM) to achieve sufficient cross-modality fusion by using single modal as the main guidance and the other modal as auxiliary. Finally, the Cross-Guided Fusion Module (CGFM) is presented and serves as the main decoding block. And each decoding block is consists of two parts with two modalities information of each being the main guidance, i.e., cross-shared Cross-Level Enhancement (CLE) and Global Auxiliary Enhancement (GAE). The main difference between the two parts is that the GFM using different modalities as the main guide. The comprehensive experimental results prove that our method achieves better performance than the state-of-the-art salient detection methods. The source code has released at: https://github.com/wangjie0825/CGFNet.git .

Visual perception plays an important role in industrial information field, especially in robotic grasping application. In order to detect the object to be grasped quickly and accurately, salient object detection (SOD) is employed to the above task. Although the existing SOD methods have achieved impressive performance, they still have some limitations in the complex interference environment of practical application. To better deal with the complex interference environment, a novel triple-modal images fusion strategy is proposed to implement SOD for robotic visual perception, namely visible-depth-thermal (VDT) SOD. Meanwhile, we build an image acquisition system under variable lighting scene and construct a novel benchmark dataset for VDT SOD (VDT-2048 dataset). Multiple modal images will be introduced to assist each other to highlight the salient regions. But, inevitably, interference will also be introduced. In order to achieve effective cross-modal feature fusion while suppressing information interference, a hierarchical weighted suppress interference (HWSI) method is proposed. The comprehensive experimental results prove that our method achieves better performance than the state-of-the-art methods.

Salient object detection (SOD) of images refers to simulating the attention mechanism of human vision to capture the most attractive objects in an image. Current SOD mainly relies on RGB images captured by optical cameras. However, existing optical cameras are not comparable to the human visual system, especially in poorly illumination scenes. Our human visual system is able to resolve scenes well in low light conditions, while optical cameras can barely image without enough illumination. To make machine vision closer to the imaging of the human eye, we propose to use thermal infrared (T) images to compensate RGB images and build a variable illumination RGBT dataset named VI-RGBT1500 for SOD. This dataset is collected under three different illumination conditions including sufficient illumination, uneven illumination and insufficient illumination to fully demonstrate the superiority of the RGBT image combination. Furthermore, we propose a multiple graph affinity interactive (MGAI) network to validate the proposed dataset. Our network structure is simple using only the MGAI to fuse the features of different modalities. Meanwhile, the MGAI model highlights valuable information during the interaction, which facilitates feature representation under variable illumination. The proposed VI-RGBT1500 dataset and three publicly available RGBT SOD datasets are used for the comparison experiments, and the results with the state-of-the-art methods prove that our VI-RGBT1500 dataset is valuable and the performance of the MGAI network is competitive. The VI-RGBT1500 dataset and the MGAI network are available at: https://github.com/huanglm-me/VI-RGBT1500.

RGB and thermal infrared (RGBT) image saliency detection is a relatively new direction in the field of computer vision. Combining the advantages of RGB images and T images can significantly improve detection performance. Currently, there are only a few methods to work on RGBT saliency detection, and the number of image samples cannot meet the training requirements for deep learning, so it remains valuable to propose an effective unsupervised method. In this paper, we present an unsupervised RGBT saliency detection method based on multi-graph fusion and learning. Firstly, RGB images and T images are adaptively fused based on boundary information to produce more accurate superpixels. Next, a multi-graph fusion model is proposed to selectively learn useful information from multi-modal images. Finally, we implement the theory of finding good neighbors in the graph affinity and propose different algorithms for two stages of saliency ranking. Experimental results on three RGBT datasets show that the proposed method is effective compared with the state-of-the-art algorithms.