Yaoyao Liu

Yaoyao Liu


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
YLYaoyao Liu
Yaoyao Liu
YLYingying Li
Yingying Li
BSBernt Schiele
Bernt Schiele
QSQianru Sun
Qianru Sun
Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. However, none of the existing CIL models can achieve the optimal tradeoff in different data-receiving settings—where typically the training-from-half (TFH) setting needs more stability, but the training-from-scratch (TFS) needs more plasticity. To this end, we design an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori. Specifically, we first introduce the key hyperparameters that influence the tradeoff, e.g., knowledge distillation (KD) loss weights, learning rates, and classifier types. Then, we formulate the hyperparameter optimization process as an online Markov Decision Process (MDP) problem and propose a specific algorithm to solve it. We apply local estimated rewards and a classic bandit algorithm Exp3 to address the issues when applying online MDP methods to the CIL protocol. Our method consistently improves top-performing CIL methods in both TFH and TFS settings, e.g., boosting the average accuracy of TFH and TFS by 2.2 percentage points on ImageNet-Full, compared to the state-of-the-art. Code is provided at https://class-il.mpi-inf.mpg.de/online/

1 November 2021 in Computer Vision and Image Understanding
Verified
Journal Article
Closed access
XLXinzhe Li
Xinzhe Li
JHJianqiang Huang
Jianqiang Huang
YLYaoyao Liu
Yaoyao Liu
QZQin Zhou
Qin Zhou
SZShibao Zheng
Shibao Zheng
QSQianru Sun
Qianru Sun

29 October 2024
Verified
Book Chapter
Closed access
PDF file
TFTom Fischer
Tom Fischer
YLYaoyao Liu
Yaoyao Liu
AJArtur Jesslen
Artur Jesslen
NANoor Ahmed
Noor Ahmed
PKPrakhar Kaushik
Prakhar Kaushik
EIEddy Ilg
Eddy Ilg

19 July 2026
Verified
Conference Paper
Open access
WCWei Cao
Wei Cao
HZHao Zhang
Hao Zhang
FTFengrui Tian
Fengrui Tian
YWYulun Wu
Yulun Wu
YLYingying Li
Yingying Li
YLYaoyao Liu
Yaoyao Liu
Camera redirection aims to replay a dynamic scene from a single monocular video under a user-specified camera trajectory. However, large-angle redirection is inherently ill-posed: a monocular video captures only a narrow spatio-temporal view of a dynamic 3D scene, providing severely limited observations of the underlying 4D world. The key challenge is therefore to recover a complete and coherent plenoptic representation from this limited input, with consistent geometry and coherent motion. While recent diffusion-based methods achieve impressive visual generation quality, they often break down under large-angle viewpoint changes far from the original trajectory, where missing visual grounding leads to severe geometric ambiguity and temporal inconsistency. To address this, we present FreeOrbit4D, an effective training-free framework that tackles this geometric ambiguity by recovering a foreground-complete 4D proxy as structural grounding for video generation. We obtain this proxy by decoupling foreground and background reconstructions: we unproject the monocular video into a static background and partial foreground point clouds in a unified global space, then leverage an object-centric multi-view diffusion model to synthesize multi-view images and reconstruct complete foreground point clouds in canonical object space. By aligning the canonical foreground point cloud to the global scene space via dense pixel-synchronized 3D–3D correspondences and projecting the foreground-complete 4D proxy onto target camera viewpoints, we provide geometric scaffolds (e.g., depth/visibility cues) that guide a conditional video diffusion model. Extensive experiments show that FreeOrbit4D produces more faithful and temporally coherent redirected videos under challenging large-angle trajectories, and our foreground-complete 4D proxy further opens a potential avenue for practical applications such as edit propagation and 4D data generation. Project page: https://freeorbit4d.vision.ischool.illinois.edu/

11 September 2025 in Neurocomputing
Verified
Journal Article
Closed access
ZLZilin Luo
Zilin Luo
ZTZichen Tian
Zichen Tian
YLYaoyao Liu
Yaoyao Liu
QSQianru Sun
Qianru Sun

1 August 2026 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Open access
JTJiapeng Tang
Jiapeng Tang
WCWei Cao
Wei Cao
BZBiao Zhang
Biao Zhang
CLChang Luo
Chang Luo
YLYaoyao Liu
Yaoyao Liu
MNMatthias Nießner
Matthias Nießner
We introduce Motion2VecSets, a 4D diffusion model for dynamic surface mesh generation from various ambiguous observations, including a sequence of RGB images, sparse and partial point clouds, and low-resolution voxel grids. While recent methods using neural field representations have shown success in modeling non-rigid objects, conventional feed-forward architectures struggle with noisy, partial, or sparse observations due to their deterministic nature. To address the inherent one-to-many mapping problem, we introduce a diffusion model that explicitly learns the shape and motion distribution of non-rigid objects through an iterative denoising process of compressed latent representations. The diffusion-based priors provide more plausible and diverse reconstructions under ambiguous conditions. Instead of relying on global latent codes, we represent 4D dynamics using latent sets. This novel 4D representation captures local shape and deformation patterns, leading to more accurate non-linear motion capture and significantly improving generalization capacity to unseen motions and identities. For temporally coherent tracking, we jointly denoise latent sets across frames and enable cross-frame information exchange. To reduce computational cost, we design an interleaved spatial-temporal attention block that alternately aggregates deformation latents along spatial and temporal dimensions. Extensive experiments on datasets of humans, animals, and articulated objects demonstrate that Motion2VecSets outperforms prior methods in reconstructing and tracking non-rigid deformations from various imperfect observations. Our implementation is available at https://vveicao.github.io/projects/Motion2VecSets/.