Yuan Rao

Dr. Yuan Rao

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
15 August 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
YRYuan Rao
Yuan Rao
WLWenjie Liu
Wenjie Liu
KLKunqian Li
Kunqian Li
HFHao Fan
Hao Fan
SWSen Wang
Sen Wang
Junyu DongJunyu Dong
Junyu Dong
Underwater images suffer from quality degradation due to the underwater light absorption and scattering. It remains challenging to enhance underwater images using deep learning-based methods since the scarcity of real-world underwater images and their enhanced counterparts. Although existing works manually select well-enhanced images as reference images to train enhancement networks in an end-to-end manner, their performance tends to be inferior in some scenarios. We argue that the manually selected reference images cannot approximate their ground truth perfectly, leading to imbalanced learning and domain shift in enhancement networks. To address this issue, we analyse widely used underwater datasets from the perspective of color spectrum distribution and surprisingly find the sound color spectrum distribution of the enhanced reference images compared to in-air datasets. Based on this perceptive observation, instead of directly learning the enhancement mapping, we propose a novel methodology to learn color compensation for general purposes. Specifically, we present a probabilistic color compensation network that estimates the probabilistic distribution of colors by multi-scale volumetric fusion of texture and color features. We further propose a novel two-stage enhancement framework that first performs color compensation and then enhancement, which is highly flexible to be integrated with an existing enhancement method without tuning. Extensive experiments on underwater image enhancement across various challenging scenarios show that our proposed approach consistently improves the results of the popular conventional and learning-based methods by a significant margin. Moreover, our enhanced images achieve superior performance on underwater salient object detection and visual 3D reconstruction, demonstrating that our method can successfully break through the generalization bottleneck of existing learning-based enhancement models. Our implementation will be made available at https://github.com/Ray2OUC/P2CNet.

9 February 2023 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
ZJZhenlin Jia
Zhenlin Jia
YRYuan Rao
Yuan Rao
HFHao Fan
Hao Fan
Junyu DongJunyu Dong
Junyu Dong
Structure-from-Motion (SfM) is a prevalent solution for vision measurement. Traditional SfM methods rely on sparse feature matching to establish visual associations between different views. However, both the incremental-based and global-based SfM show sensitiveness to outliers generated by existing hand-crafted features under challenging environments. Planar marker-assisted SfM is an alternative solution that employs artificial landmarks to help robust pose estimation. Nevertheless, using planar markers also exists the planar pose ambiguity problem, which means there may be more than one reasonable solution for pose estimation; it is a great challenge for camera pose estimation. This paper proposes an efficient planar marker-assisted global SfM method and resolves the planar pose ambiguity problem by examining the rotation consistency across multiple views. In particular, we construct two complementary graphs and present a graph filter method to filter out the views with pose ambiguity according to our designed rotation consistency checking. Afterward, we perform sparse feature matching following the proposed graph structure and make a pose-fixed bundle adjustment for high-quality 3D reconstruction. Extensive experiments have been conducted on synthetic and real datasets, and the results verify the effectiveness of the proposed method. Our method outperforms several state-of-the-art methods with encouraging efficiency, especially in challenging scenes.

3 March 2023 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
YRYuan Rao
Yuan Rao
Yakun JuYakun Ju
Yakun Ju
SWSen Wang
Sen Wang
Feng GaoFeng Gao
Feng Gao
HFHao Fan
Hao Fan
Junyu DongJunyu Dong
Junyu Dong
Recent feature descriptor research has witnessed tremendous progress with the development of the deep neural network. However, most existing descriptors solely focus on learning strong discriminativeness with deep-invariant features, neglecting their representation ability and rich hierarchical clues hidden in images, which could further establish high-quality matches via implicit hierarchical comparisons in rich representative descriptor space. In this article, we consider both the discriminative and representation ability of feature descriptors to enrich the descriptor space with a novel representative learning framework. On the one hand, we introduce histogram of oriented gradient (HOG) as a prior term to guide our descriptor to learn a powerful representation and robustness in a self-supervised manner. On the other hand, we present an adaptive triplet loss (ATL), which penalizes the triplet loss (TL) according to the descriptor matching distances in order to encourage our descriptor to learn strong discriminativeness. Moreover, to fully use the information encapsulated in images and boost the representation ability, we propose a novel HIerarchical Feature Transformer Network (HIFT), which derives dense descriptions from the semantic and cross-scale-enhanced hierarchical features in a local-to-global manner. Extensive experiments on popular feature matching and visual localization benchmarks show that the HIFT achieves highly competitive performance compared with the state-of-the-art methods. Applications on visual measurement tasks of visual 3-D reconstruction and ego-motion estimation also demonstrate the high generalization ability of our method. Our model is available at https://github.com/Ray2OUC/HIFT .

14 April 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
YRYuan Rao
Yuan Rao
Yakun JuYakun Ju
Yakun Ju
CLCong Li
Cong Li
EREric Rigall
Eric Rigall
JYJian Yang
Jian Yang
Junyu DongJunyu Dong
Junyu Dong
Recent advances on feature descriptors for image matching put more emphasis on encoding invariances ( e.g . illumination invariance) to promote the descriptors’ discriminative power. However, according to the information entropy, more invariance implies greater certainty and less informativeness in a descriptor. Consequently, descriptors encoding too many invariances usually show poor generalization to unknown image changes, lacking enough informativeness to cover the large uncertainty in unseen scenes. This limits the application scenarios of learned descriptors. In this paper, we propose to alleviate this issue from the perspective of informativeness and we thus design hierarchical consistent constraint by introducing regression feedback in a self-supervised manner. Combined with the hardest-within-batch matching constraint, we form a novel dual supervision framework, to encourage the descriptor to learn an informative representation while maintaining a good discriminative power. Moreover, to fully mine the context information hidden in image and boost the informativeness in turn, we present AANet, a descriptor network that efficiently predicts dense description by the powerful Attentional Aggregation of multi-level features. Experiments across challenging feature matching on HPatches, RDNIM datasets, and visual localization tasks on Aachen Day-night dataset show that our method outperforms recent state-of-the-art descriptors while keeping encouraging efficiency. The application of visual 3D reconstruction on various scenarios also demonstrates the high generalization ability of our method.

22 April 2022 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
YRYuan Rao
Yuan Rao
JYJian Yang
Jian Yang
Yakun JuYakun Ju
Yakun Ju
CLCong Li
Cong Li
EREric Rigall
Eric Rigall
Junyu DongJunyu Dong
Junyu Dong
Learning-based feature descriptors have been dominantly popular for their notable performance on feature matching tasks along with the rapid development of convolutional neural networks (CNNs). However, existing popular learning-based methods predict discriminative description solely using the high-level features from the last layer of deep CNNs while neglecting the rich complementary clues hidden in intermediary multilevel features, which could further promote the discriminative power by introducing the implicit hierarchical comparison into descriptor space. This hinders the optimization of learned descriptors and limits their performance on real-world visual measurement tasks. In this regard, we propose hierarchical view consistency (HVC) for fully leveraging the complementary information of multilevel features. Specifically, we first present a novel multiviewer neural network (MVNet), which benefits from multiple viewers with local-to-global receptive fields and efficiently generates dense descriptions in a coarse-to-fine manner. Next, we introduce the HVC, i.e., ensuring consistent yet diverse hierarchical features between views, to encourage viewers to encode as hierarchical features as possible while increasing the hierarchical similarity for reliable matches. With our proposed triplet training strategy, MVNet leverages the rich hierarchical complementary clues in multilevel features and efficiently predicts strong discriminative descriptions. Our experiments on feature matching and challenging visual measurement tasks of visual localization and visual 3-D reconstruction demonstrate that our proposed descriptor is efficient and generalizes well to various scenarios.

24 March 2023 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
YLYimei Liu
Yimei Liu
YRYuan Rao
Yuan Rao
EREric Rigall
Eric Rigall
HFHao Fan
Hao Fan
Junyu DongJunyu Dong
Junyu Dong
Multi-view stereo (MVS) aims to measure the precise surface depth of a scene from observations at multiple photography angles and then densely reconstruct its 3D geometry information. Learning-based MVS approaches have been dominantly popular for their robustness to low texture areas and non-Lambertian surfaces. However, most existing methods focus on estimating depth maps for input images by constructing global cost volumes and designing ingenious yet large variance-based 3D-CNN networks for cost volume regularization. Such approaches ignore the co-visible relation embedded in multiple views, resulting in heavy computation, erroneous cost aggregation from invisible views, and finally inaccurate 3D reconstruction results. In this paper, we propose a Co-visibility Reasoning Multi-View Stereo Network (CR-MVSNet) to explore the co-visible relations hidden in multiple views for reliable multi-view similarity measurement and efficient reconstruction. Precisely, the proposed Co-visibility Reasoning Cost Aggregation (CRCA) module includes the adaptive inter-cost volume aggregation via mining the uncertainty of co-visibility relations in multiple views and the adaptive intra-cost volume aggregation by exploiting spatial contextual information. Moreover, the cost volumes are constructed via the proposed global-to-patch manner to speed up computation. Experimental results show that our approach achieves the best overall performance on the DTU, Tanks and Temples, and ETH3D-test datasets over recent state-of-the-art MVS algorithms. The consistently favorable results on three datasets with completely different depth ranges proved the superiority and generalizability of CR-MVSNet.