21 June 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Salient object detection and camouflaged object detection have attracted increasing attention due to their significant practical applications. While these two domains share similarities in recognition methods and object characteristics, they also exhibit distinctions. In this paper, we propose a novel multi-view guided network for camouflaged and salient object detection, utilizing the Transformer as the backbone network for feature extraction. Capitalizing on shared characteristics, we introduce a CNN-based multi-view encoder and a multi-view fusion module, enhancing the acquisition of multi-perspective information while minimizing the increase in computational cost. Moreover, recognizing domain differences, we incorporate an attention exploration module, seamlessly integrating multi-view features with globally extracted features from the backbone network. This integration involves simultaneous exploration from both positional and color perspectives, unearthing valuable information to identify salient and camouflaged objects. Our approach maximizes shared characteristics between the two tasks while effectively addressing their differences, leading to precise object identification—be it for camouflaged or salient objects. Extensive experiments on nine challenging benchmark datasets demonstrate the superior performance of our method across four widely used evaluation metrics, outperforming 34 state-of-the-art methods. Furthermore, we applied our method to other visually-related tasks, such as polyp segmentation and defect detection. The results further demonstrate the versatility of our model. The source code and results of our method are available at https://github.com/1900zpf/MVGNet.
7 September 2023 in IEEE Transactions on Multimedia
Recent studies have shown that joint depth and pose estimation using convolutional neural networks (CNNs) can learn unlabelled monocular frames. However, three problems remain: 1) CNNs can only extract local features due to the limited receptive field, 2) scale ambiguity is inherent in the monocular task, and 3) illness regions violate the photometric consistency assumption and produce large errors. We propose a novel framework, ADPDepth, with corresponding effective strategies to ameliorate the above problems. First, a PCAtt module is designed to capture the correlation between channels and efficiently extract multiscale spatial information using a multibranch parallel strategy. Second, depth-pose consistency loss is proposed based on the geometric consistency in depth and pose to constrain the scale between samples, eliminate scale ambiguity and obtain a globally consistent scale. To further improve performance, a cover mask is derived from depth-pose consistency for filtering dynamic objects and outliers to reduce the adverse effects of these illness regions. Extensive experiments are conducted on the KITTI, NYU-Depth and Make3D datasets. Based on public benchmarks, the experimental results confirm that the proposed ADPDepth framework achieves state-of-the-art performance. The effectiveness of each strategy is also verified in subsequent ablation experiments.
7 September 2023 in IEEE Transactions on Circuits and Systems for Video Technology
In recent years, Transformers have been gradually applied in salient object detection tasks with good results. However, the Transformer’s global modeling capabilities can lead to the loss of local details that are important in salient object detection tasks. A feature extraction backbone based on a convolutional neural network (CNN) is good at extracting local detail features due to the gradual expansion of the receptive field but is limited by the size of the receptive field, resulting in an insufficient ability to extract global semantic features. Therefore, this paper combines the Transformer with a CNN and presents a dual-branch encoder to ensure that the features extracted contain rich global semantic information as well as local detail features. In addition, due to the different features extracted by the Transformer and CNN, noise may be introduced in the fusion of the two features, so different features need to be processed correspondingly during fusion. The fusion enhancement module (FEM) we propose fuses the features of the two branches step by step. A hybrid attention mechanism is used to carry out weighted fusion of different features. This progressive approach minimizes the differences between the features of the two branches so that the merged features retain the semantic and detail features extracted by the two branches to the greatest extent. Considering the loss of detailed information caused by repeated downsampling, we propose an edge refinement module (ERM) to address the need for accurate outline prediction. This module leverages salient features to obtain edge features and gradually refines the prediction results by incorporating these edge features. It makes full use of the connection between salient features and edge features and does not introduce additional edges to extract branches. Extensive experimental evaluations conducted on five benchmark tests demonstrate the superior performance of our method compared to other existing approaches. Code can be found at https://github.com/gfq1605694825/DSRNet-main.
8 May 2023 in Computers & Graphics
22 November 2023 in IEEE Robotics and Automation Letters
In the field of 3D object detection, voxel-based methods are the most commonly used and exhibit high accuracy. However, point-based networks, which have the capability to preserve the original point features, are unable to surpass the accuracy achieved by voxel-based methods. We observed that during downsampling, too many useless background points are retained in the prevailing farthest point sample, which interferes with the accuracy of the point-based network and introduces redundant computation. To address this issue, in this letter, a point-based single-stage 3D detection network called PSNS-SSD is designed, and its accuracy rivals the accuracy of the two-stage voxel-based methods. Our method leverages the semantic information that suppresses nonsalient features used for guidance and discards background points during the downsampling process. In addition, we propose two modules, namely, linear-conv channel-enhanced self-attention for point cloud feature extraction and global-local coupling attention for center prediction, to enhance the performance and efficiency of our model. Our PSNS-SSD model is extensively tested on the most widely available 3D dataset—the KITTI dataset. The experimental results validate the three proposed modules in terms of accuracy for small object detection and speed, and the model achieves accuracy levels that are on par with the accuracy levels of cutting-edge voxel-based techniques. Moreover, our detector can achieve a speed of 70 FPS on the KITTI dataset, making it a promising candidate for practical applications.
7 April 2025 in Neural Networks