Rong Xie

Rong Xie

Auto-generated profile, based on publicly available information

Rong Xie

School of Information Science and Electronic Engineering, Shanghai Jiao Tong University, Shanghai, China

Research Focus & Metrics

Academic Career Length

19Years (2008-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 October 2023
Verified
Conference Paper
Open access
JWJionghao Wang
Jionghao Wang
ZCZiyu Chen
Ziyu Chen
JLJun Ling
Jun Ling
RXRong Xie
Rong Xie
LSLi Song
Li Song
360° panoramas are extensively utilized as environmental light sources in computer graphics. However, capturing a 360° × 180° panorama poses challenges due to the necessity of specialized and costly equipment, and additional human resources. Prior studies develop various learning-based generative methods to synthesize panoramas from a single Narrow Field-of-View (NFoV) image, but they are limited in alterable input patterns, generation quality, and controllability. To address these issues, we propose a novel pipeline called PanoDiff, which efficiently generates complete 360° panoramas using one or more unregistered NFoV images captured from arbitrary angles. Our approach has two primary components to overcome the limitations. Firstly, a two-stage angle prediction module to handle various numbers of NFoV inputs. Secondly, a novel latent diffusion-based panorama generation model uses incomplete panorama and text prompts as control signals and utilizes several geometric augmentation schemes to ensure geometric properties in generated panoramas. Experiments show that PanoDiff achieves state-of-the-art panoramic generation quality and high controllability, making it suitable for applications such as content editing.

29 November 2024
Verified
Book Chapter
Closed access
PDF file
JWJionghao Wang
Jionghao Wang
YLYuan Liu
Yuan Liu
ZDZhiyang Dou
Zhiyang Dou
ZYZhengming Yu
Zhengming Yu
YLYongqing Liang
Yongqing Liang
WWWenping Wang
Wenping Wang

8 April 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
SGShuai Guo
Shuai Guo
QWQiuwen Wang
Qiuwen Wang
YGYijie Gao
Yijie Gao
RXRong Xie
Rong Xie
LLLin Li
Lin Li
LSLi Song
Li Song
Novel-view synthesis with sparse input views is important for practical applications such as AR/VR and autonomous driving. Many works in this field have already integrated depth information into NeRF, utilizing depth priors for assistance in geometric and spatial understanding. However, most existing work tends to either overlook the inaccuracies in depth maps or only handle them roughly, limiting the effectiveness of the synthesis. To address this issue, we propose a depth-guided robust point cloud fusion NeRF for sparse input synthesis. We first construct a point cloud for each input view, with a novel point cloud representation based on learnable matrices and vectors. Then, through an additional lightweight scene fusion network, we fuse the point clouds from each input view to build a point cloud of the entire scene. By optimizing the point cloud representation and scene fusion network, inaccuracies in the depth map can be adjusted and refined, thereby achieving a more precise perception of the overall scene. Each voxel in the scene is determined by referencing the fused point cloud to establish its density and appearance. Experimental results demonstrate that our method outperforms state-of-the-art baselines.

18 April 2024 in Displays
Verified
Journal Article
Closed access
YZYuheng Zhang
Yuheng Zhang
JWJia Wang
Jia Wang
YZYucheng Zhu
Yucheng Zhu
RXRong Xie
Rong Xie

13 January 2022 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
CLChen Li
Chen Li
LSLi Song
Li Song
SCShuai Chen
Shuai Chen
RXRong Xie
Rong Xie
WZWenjun Zhang
Wenjun Zhang
In this paper, we propose a deep learning based sensor-driven method for online video stabilization. This method utilizes euler angle and acceleration estimated from the gyroscope and accelerator to assist stable video reconstruction. We introduce two simple sub-networks for trajectory optimization, the first network exploits real unstable trajectory and acceleration of camera to detect shooting scenarios. This network also generates an attention mask to choose scenario-specific features adaptively. Then the second network predicts smooth camera paths based on real unstable trajectory using LSTM under the supervision of the former mask. The output of trajectory optimization network is filtered with a two-step modification process to guarantee the smoothness. The real and smoothed camera paths are then utilized as guidance to projectively generate stable frames. We also capture videos with sensor data covering seven typical shooting scenarios, and design a groundtruth generation method to construct pseud-labels. Moreover, the trajectory smoothing network allows 3 or 10 frames buffer as the future information to construct a lookahead filter. Experimental results show that our online method could outperform other offline state-of-the-art methods in several shaky video clips with fewer buffer frames over both general and low-quality videos. Furthermore, our method could effectively reduce running times without image content analysis, and the stabilization efficiency reaches 25 fps on 1080p videos.

1 October 2023
Verified
Conference Paper
Closed access
YWYunqian Wen
Yunqian Wen
Bo LiuBo Liu
Bo Liu
JCJingyi Cao
Jingyi Cao
RXRong Xie
Rong Xie
LSLi Song
Li Song
Face de-identification involves concealing the true identity of a face while retaining other facial characteristics. Current target-generic methods typically disentangle identity features in the latent space, using adversarial training to balance privacy and utility. However, this pattern often leads to a trade-off between privacy and utility, and the latent space remains difficult to explain. To address these issues, we propose IDeudemon, which employs a "divide and conquer" strategy to protect identity and preserve utility step by step while maintaining good explainability. In Step I, we obfuscate the 3D disentangled ID code calculated by a parametric NeRF model to protect identity. In Step II, we incorporate visual similarity assistance and train a GAN with adjusted losses to preserve image utility. Thanks to the powerful 3D prior and delicate generative designs, our approach could protect the identity naturally, produce high quality details and is robust to different poses and expressions. Extensive experiments demonstrate that the proposed IDeudemon outperforms previous state-of-the-art methods.