Ruixuan Cong

Ruixuan Cong

Auto-generated profile, based on publicly available information

Ruixuan Cong

State Key Laboratory of Virtual Reality Technology and Systems, School of Computer Science and Engineering, Beihang University, Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 November 2024 in IEEE Transactions on Instrumentation and Measurement
Verified
Journal Article
Closed access
TWTun Wang
Tun Wang
HSHao Sheng
Hao Sheng
RCRongshan Chen
Rongshan Chen
RCRuixuan Cong
Ruixuan Cong
MZMingyuan Zhao
Mingyuan Zhao
ZCZhenglong Cui
Zhenglong Cui
Light field (LF) technology captures information from multiple directions and angles, enabling precise disparity estimation. Recently, matching cost-based approaches have advanced rapidly and shown satisfactory results. However, these methods typically depend on fixed disparity candidates, leading to inadequate utilization of candidates and making them unsuitable for LF scenes with varying baselines. Multi-direction line structures of epipolar-plane images (EPIs) associate multiple viewpoints, adaptively perceiving disparity ranges and accurately matching features in real scenes. In this paper, we propose an Adaptive EPI-Matching Cost for LF disparity estimation, which is proven to enhance the adaptability across datasets with varying baselines. Our approach calculates pixel-level disparity candidates to keep the predicted distribution near the ground truth and matches line structures to improve accuracy. Then, to enhance robustness during the adaptive process, we introduce an intra-EPI extraction module that dynamically establishes correlations in the local EPI while supplementing spatial information. Finally, we present a network named Adaptive EPI-Matching Cost Network (AEMCNet) for LF disparity estimation. Experimental results demonstrate that AEMCNet achieves state-of-the-art performance and robustness on various LF datasets with different baselines. Specifically, on the Sparse Light Field Dataset, our method reduces the mean square error by 49.6%.

30 June 2022 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Open access
HSHao Sheng
Hao Sheng
RCRuixuan Cong
Ruixuan Cong
DYDa Yang
Da Yang
RCRongshan Chen
Rongshan Chen
SWSizhe Wang
Sizhe Wang
ZCZhenglong Cui
Zhenglong Cui
As one of the fundamental technologies for scene understanding, semantic segmentation has been widely explored in the last few years. Light field cameras encode the geometric information by simultaneously recording the spatial information and angular information of light rays, which provides us with a new way to solve this issue. In this paper, we propose a high-quality and challenging urban scene dataset, containing 1074 samples composed of real-world and synthetic light field images as well as pixel-wise annotations for 14 semantic classes. To the best of our knowledge, it is the largest and the most diverse light field dataset for semantic segmentation. We further design two new semantic segmentation baselines tailored for light field and compare them with state-of-the-art RGB, video and RGB-D-based methods using the proposed dataset. The outperforming results of our baselines demonstrate the advantages of the geometric information in light field for this task. We also provide evaluations of super-resolution and depth estimation methods, showing that the proposed dataset presents new challenges and supports detailed comparisons among different methods. We expect this work inspires new research direction and stimulates scientific progress in related fields. The complete dataset is available at https://github.com/HAWKEYE-Group/UrbanLF.

2 June 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
RCRuixuan Cong
Ruixuan Cong
HSHao Sheng
Hao Sheng
DYDa Yang
Da Yang
ZCZhenglong Cui
Zhenglong Cui
RCRongshan Chen
Rongshan Chen
Global context information is particularly important for comprehensive scene understanding. It helps clarify local confusions and smooth predictions to achieve fine-grained and coherent results. However, most existing light field processing methods leverage convolution layers to model spatial and angular information. The limited receptive field restricts them to learn long-range dependency in LF structure. In this paper, we propose a novel network based on deep efficient transformers ( i.e., LF-DET) for LF spatial super-resolution. It develops a spatial-angular separable transformer encoder with two modeling strategies termed as sub-sampling spatial modeling and multi-scale angular modeling for global context interaction. Specifically, the former utilizes a sub-sampling convolution layer to alleviate the problem of huge computational cost when capturing spatial information within each sub-aperture image. In this way, our model can cascade more transformers to continuously enhance feature representation with limited resources. The latter processes multi-scale macro-pixel regions to extract and aggregate angular features focusing on different disparity ranges to well adapt to disparity variations. Besides, we capture strong similarities among surrounding pixels by dynamic positional encodings to fill the gap of transformers that lack of local information interaction. The experimental results on both real-world and synthetic LF datasets confirm our LF-DET achieves a significant performance improvement compared with state-of-the-art methods. Furthermore, our LF-DET shows high robustness to disparity variations through the proposed multi-scale angular modeling.

22 May 2023 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
HSHao Sheng
Hao Sheng
SWSizhe Wang
Sizhe Wang
DYDa Yang
Da Yang
RCRuixuan Cong
Ruixuan Cong
ZCZhenglong Cui
Zhenglong Cui
RCRongshan Chen
Rongshan Chen
Compared with external-supervised learning-based (ESLB) methods, self-supervised learning-based (SSLB) methods can overcome the domain gap problem caused by different light field (LF) acquisition conditions, which results in the performance degradation of light field super-resolution on unseen test datasets. Current SSLB methods exploit the cross-scale recurrence feature in the single view image for super-resolution, ignoring the correlation information among views. Different from previous works, we propose a cross-view recurrence-based self-supervised mapping framework to correlate complementary information among views in the down-scaled input LF. Specifically, the cross-view recurrence information consists of geometry structure features and similar structure features. The former is to provide sub-pixel information according to disparity correlations among adjacent views, and the latter is to acquire similar color and contour information among arbitrary views, which can compensate for error disparity guidance of geometry structure features in sharp variance areas. Moreover, instead of the widely used “All-to-All” strategy, we propose a “Part-to-Part” mapping strategy, which is better competent for SSLB approaches with limited training examples solely extracted from input LF. Finally, considering that self-supervised methods need to retrain from the beginning toward each test image, based on the proposed “part-to-part” strategy, an efficient end-to-end network is designed to extract these cross-view features for superior SASR performance with less training time. Experiment results demonstrate that our method outperforms other state-of-the-art ESLB methods on both large and small domain gap cases. Compared with the only SSLB method (LFZSSR [1]), our approach achieves better performance with 524 times less training time.

14 December 2023 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
JSJiahao Shen
Jiahao Shen
HSHao Sheng
Hao Sheng
shuai wangshuai wang
shuai wang
RCRuixuan Cong
Ruixuan Cong
DYDa Yang
Da Yang
Yang ZhangYang Zhang
Yang Zhang
With the development of smart cities, video surveillance has become more prevalent in urban areas. The rapid growth of data brings challenges to video processing and analysis. Multi-object tracking (MOT), one of the most fundamental tasks in computer vision, has a wide range of applications and development prospects. MOT aims to locate multiple objects and maintain their unique identities by analyzing the video frame by frame. Most existing MOT frameworks are deployed in centralized systems, which are convenient for management but have problems such as weak algorithm adaptability, limited system scalability, and poor data security. In this paper, we propose a distributed MOT algorithm based on multi-agent reinforcement learning (DMARL-Tracker), which formulates MOT as a Markov decision process (MDP). Each object adjusts its tracking strategy during interactions with the environment. The benchmark results on MOT17 and MOT20 prove that our proposed algorithm achieves state-of-the-art (SOTA) performance. Based on this, we further integrate DMARL-Tracker into the blockchain and propose a blockchain-based collaborative MOT framework. All nodes collaborate and share information through the blockchain, achieving adaptation in different complex scenarios while ensuring data security. The simulation results show that our framework achieves good performance in terms of tracking and resource consumption.

14 December 2023 in IEEE Transactions on Computers
Verified
Journal Article
Closed access
DYDa Yang
Da Yang
ZCZhenglong Cui
Zhenglong Cui
HSHao Sheng
Hao Sheng
RCRongshan Chen
Rongshan Chen
RCRuixuan Cong
Ruixuan Cong
ZXZhang Xiong
Zhang Xiong
Stereo vision is widely studied for depth information extraction. However, occlusion and noise pose significant challenges to traditional methods due to failure in photo consistency. In this paper, an occlusion and noise-aware stereo framework named ONAF is proposed to get a robust depth estimation by integrating the advantages of correspondence cues and refocusing cues from light field (LF). ONAF consists of two special depth cue extractors: correspondence depth cue extractor (CCE) and refocusing depth cue extractor (RCE). CCE extracts accurate correspondence depth cues in occlusion areas based on multi-direction Ray-Epipolar Plane Images (Ray-EPIs) from LF, which are more robust than traditional multi-direction EPIs. RCE generates accurate refocusing depth cues in noise areas, benefitting from the many-to-one integration strategy and the directional perception of texture and occlusion based on multi-direction focal stacks from LF. Attention mechanism is introduced to complementarily fuse CCE and RCE to generate optimum depth maps. The experimental results prove the effectiveness of ONAF, which outperforms state-of-the-art disparity estimation methods, especially in occlusion and noise areas.