Object SLAM is considered increasingly significant for robot high-level perception and decision-making. Existing studies fall short in terms of data association, object representation, and semantic mapping and frequently rely on additional assumptions, limiting their performance. In this article, we present a comprehensive object SLAM framework that focuses on object-based perception and object-oriented robot tasks. First, we propose an ensemble data association approach for associating objects in complicated conditions by incorporating parametric and nonparametric statistic testing. In addition, we suggest an outlier-robust centroid and scale estimation algorithm for modeling objects based on the iForest and line alignment. Then a lightweight and object-oriented map is represented by estimated general object models. Taking into consideration the semantic invariance of objects, we convert the object map to a topological map to provide semantic descriptors to enable multimap matching. Finally, we suggest an object-driven active exploration strategy to achieve autonomous mapping in the grasping scenario. A range of public datasets and real-world results in mapping, augmented reality, scene matching, relocalization, and robotic manipulation have been used to evaluate the proposed object SLAM framework for its efficient performance.
24 April 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Existing lightweight salient object detection (SOD) methods aim to solve the problem of high computational costs that is prevalent with heavyweight methods. However, compared with heavyweight methods, the detection accuracy of lightweight methods is greatly reduced while real-time performance is not significantly improved. Therefore, we aim to establish a trade off between computational cost and detection performance by improving the network efficiency. We propose a fast and extremely lightweight end-to-end wavelet neural network (ELWNet) for real-time salient object detection. ELWNet can achieve salient object detection and segmentation at approximately 70FPS (GPU), 19FPS (CPU) with 76K parameters and 0.38G FLOPs. We introduce wavelet transform theory into a neural network, proposing a wavelet transform module (WTM), a wavelet transform fusion module (WTFM), a novel feature residual mechanism, and construct an efficient architecture. The wavelet transform theory is integrated into the neural network to realize the interaction between the features in the frequency and the time domain. Meanwhile, ELWNet does not rely on a pre-trained model, which significantly reduces redundant features. We validate the performance of ELWNet using five well-known datasets, and demonstrate state-of-the-art performance compared with 24 other SOD models in terms of being lightweight, detection accuracy and real-time capabilities. Our method maintains high detection performance while reducing the number of model parameters by approximately 99% compared with heavyweight methods.
Untreated colorectal polyps can develop into colorectal cancer, which is a leading cause of cancer-related deaths. Colonoscopy is a commonly-used method for colorectal polyp scanning, but limited to the experience and subjectivity of clinicians, one out of four polyps cannot be correctly recognized. In this paper, we propose an automatic colorectal polyp segmentation system based on the deep convolutional neural network, aiming to improve the accuracy of colorectal polyp scanning. The proposed ABC-Net is comprised of a shared encoder and two novel mutually-constrained decoders for simultaneous polyp area and boundary segmentation. To sufficiently exploit multi-scale image information, the selective feature modules are embedded into the network and used for dynamically learning and fusing multi-scale feature representations. Furthermore, a new boundary-sensitive loss is proposed to model the interdependencies between the area and boundary branches, the information of the two branches are reciprocally propagated and constrained, yielding a significant improvement in segmentation accuracy. Extensive experiments are conducted on three public colorectal polyp datasets, and the results, e.g., F1 scores are 0.866, 0.915, 0.874 in EndoScene, Kvasir-SEG, and ETIS-Larib datasets, demonstrate the advantages of the proposed method.
14 January 2021 in IEEE Transactions on Industrial Electronics
Autonomous elevator operation is considered a promising solution for mobile navigation in office buildings. As a fundamental function, elevator button recognition remains unsolved due to the challenging image conditions and severe data imbalance problem. In this article, we propose an accurate and efficient framework, named OCR-RCNN, for elevator button recognition. The framework is comprised of an region-based convolutional neural network (R-CNN)-based button detector and an attention-RNN-based character recognizer. Leveraging the two components, we further propose an end-to-end architecture and a cascaded architecture to explore the most effective network design for the framework. Moreover, a perspective distortion removal algorithm is also developed to enhance the inference performance of OCR-RCNN. Another key contribution of this work is that we release the first large-scale elevator panel dataset with 2005 images and 21 767 button labels. Extensive experiments are conducted on the released dataset and other two publicly available datasets. The proposed framework achieves an F1 score of 0.94, 1.00, and 1.00 in detection task, and an accuracy of 79.6% 96.5%, and 96.4% in character recognition task. The results demonstrate the advantages of our method, outperforming alternative strategies and other state-of-the-art methods in the literature. The data and code are available on the project webpage https://github.com/zhudelong/ocr-rcnn-v2 .
23 June 2020 in IEEE Transactions on Automation Science and Engineering
Registration is an essential problem in image-guided surgery (IGS) since it brings different involved coordinate frames together. Nonrigid or deformable registration still faces many challenges, such as two point sets (PSs) are partially overlapped. To tackle the challenges in the nonrigid registration, we introduce a new two-step point-based registration pipeline that includes two steps. In the first step, the rigid transformation between the two spaces is recovered where the orientation vectors are adopted. In the second step, built on the nonrigid coherent point drift (CPD) approach, the anisotropic positional noise is also assumed. Registration results on the human liver verify the proposed approach' great improvements over the other methods. First, the rotation and translation are recovered with smaller error values than the existing methods. Second, our registration method's performance is much more robust to the partial overlapping between two PSs. Third, the two-step registration framework achieves the best performances in most test cases when there is a localization error in acquiring the intraoperative data.