Yunrong Guo

Yunrong Guo

Auto-generated profile, based on publicly available information

Yunrong Guo

NVIDIA, Canada

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 April 2023 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
MMMayank Mittal
Mayank Mittal
CYCalvin Yu
Calvin Yu
QYQinxi Yu
Qinxi Yu
JLJingzhou Liu
Jingzhou Liu
NRNikita Rudin
Nikita Rudin
AGAnimesh Garg
Animesh Garg
We present Orbit , a unified and modular framework for robot learning powered by Nvidia Isaac Sim. It offers a modular design to easily and efficiently create robotic environments with photo-realistic scenes and high-fidelity rigid and deformable body simulation. With Orbit , we provide a suite of benchmark tasks of varying difficulty– from single-stage cabinet opening and cloth folding to multi-stage tasks such as room reorganization. To support working with diverse observations and action spaces, we include fixed-arm and mobile manipulators with different physically-based sensors and motion generators. Orbit allows training reinforcement learning policies and collecting large demonstration datasets from hand-crafted or expert solutions in a matter of minutes by leveraging GPU-based parallelization. In summary, we offer an open-sourced framework that readily comes with 16 robotic platforms, 4 sensor modalities, 10 motion generators, more than 20 benchmark tasks, and wrappers to 4 learning libraries. With this framework, we aim to support various research areas, including representation learning, reinforcement learning, imitation learning, and task and motion planning. We hope it helps establish interdisciplinary collaborations in these communities, and its modularity makes it easily extensible for more tasks and applications in the future. For videos, documentation, and code: https://isaac-orbit.github.io/ .

19 November 2024 in ACM Transactions on Graphics
Verified
Journal Article
Open access
CTChen Tessler
Chen Tessler
YGYunrong Guo
Yunrong Guo
ONOfir Nabati
Ofir Nabati
GCGal Chechik
Gal Chechik
XPXue Bin Peng
Xue Bin Peng
Crafting a single, versatile physics-based controller that can breathe life into interactive characters across a wide spectrum of scenarios represents an exciting frontier in character animation. An ideal controller should support diverse control modalities, such as sparse target keyframes, text instructions, and scene information. While previous works have proposed physically simulated, scene-aware control models, these systems have predominantly focused on developing controllers that each specializes in a narrow set of tasks and control modalities. This work presents MaskedMimic, a novel approach that formulates physics-based character control as a general motion inpainting problem. Our key insight is to train a single unified model to synthesize motions from partial (masked) motion descriptions, such as masked keyframes, objects, text descriptions, or any combination thereof. This is achieved by leveraging motion tracking data and designing a scalable training method that can effectively utilize diverse motion descriptions to produce coherent animations. Through this process, our approach learns a physics-based controller that provides an intuitive control interface without requiring tedious reward engineering for all behaviors of interest. The resulting controller supports a wide range of control modalities and enables seamless transitions between disparate tasks. By unifying character control through motion inpainting, MaskedMimic creates versatile virtual characters. These characters can dynamically adapt to complex scenes and compose diverse motions on demand, enabling more interactive and immersive experiences.

26 July 2023 in ACM Transactions on Graphics
Verified
Journal Article
Open access
HZHaotian Zhang
Haotian Zhang
YYYe Yuan
Ye Yuan
VMViktor Makoviychuk
Viktor Makoviychuk
YGYunrong Guo
Yunrong Guo
SFSanja Fidler
Sanja Fidler
KFKayvon Fatahalian
Kayvon Fatahalian
We present a system that learns diverse, physically simulated tennis skills from large-scale demonstrations of tennis play harvested from broadcast videos. Our approach is built upon hierarchical models, combining a low-level imitation policy and a high-level motion planning policy to steer the character in a motion embedding learned from broadcast videos. When deployed at scale on large video collections that encompass a vast set of examples of real-world tennis play, our approach can learn complex tennis shotmaking skills and realistically chain together multiple shots into extended rallies, using only simple rewards and without explicit annotations of stroke types. To address the low quality of motions extracted from broadcast videos, we correct estimated motion with physics-based imitation, and use a hybrid control policy that overrides erroneous aspects of the learned motion embedding with corrections predicted by the high-level policy. We demonstrate that our system produces controllers for physically-simulated tennis players that can hit the incoming ball to target positions accurately using a diverse array of strokes (serves, forehands, and backhands), spins (topspins and slices), and playing styles (one/two-handed backhands, left/right-handed play). Overall, our system can synthesize two physically simulated characters playing extended tennis rallies with simulated racket and ball dynamics. Code and data for this work is available at https://research.nvidia.com/labs/toronto-ai/vid2player3d/.

23 July 2023
Verified
Conference Paper
Open access
MHMohamed Hassan
Mohamed Hassan
YGYunrong Guo
Yunrong Guo
TWTingwu Wang
Tingwu Wang
MBMichael Black
Michael Black
SFSanja Fidler
Sanja Fidler
XPXue Bin Peng
Xue Bin Peng

23 July 2023
Verified
Conference Paper
Open access
CTChen Tessler
Chen Tessler
YKYoni Kasten
Yoni Kasten
YGYunrong Guo
Yunrong Guo
SMShie Mannor
Shie Mannor
GCGal Chechik
Gal Chechik
XPXue Bin Peng
Xue Bin Peng
In this work, we present Conditional Adversarial Latent Models (CALM), an approach for generating diverse and directable behaviors for user-controlled interactive virtual characters. Using imitation learning, CALM learns a representation of movement that captures the complexity and diversity of human motion, and enables direct control over character movements. The approach jointly learns a control policy and a motion encoder that reconstructs key characteristics of a given motion without merely replicating it. The results show that CALM learns a semantic motion representation, enabling control over the generated motions and style-conditioning for higher-level task training. Once trained, the character can be controlled using intuitive interfaces, akin to those found in video games.

13 July 2024
Verified
Conference Paper
Open access
JJJordan Juravsky
Jordan Juravsky
YGYunrong Guo
Yunrong Guo
SFSanja Fidler
Sanja Fidler
XPXue Bin Peng
Xue Bin Peng
Physically-simulated models for human motion can generate high-quality responsive character animations, often in real-time. Natural language serves as a flexible interface for controlling these models, allowing expert and non-expert users to quickly create and edit their animations. Many recent physics-based animation methods, including those that use text interfaces, train control policies using reinforcement learning (RL). However, scaling these methods beyond several hundred motions has remained challenging. Meanwhile, kinematic animation models are able to successfully learn from thousands of diverse motions by leveraging supervised learning methods. Inspired by these successes, in this work we introduce SuperPADL, a scalable framework for physics-based text-to-motion that leverages both RL and supervised learning to train controllers on thousands of diverse motion clips. SuperPADL is trained in stages using progressive distillation, starting with a large number of specialized experts using RL. These experts are then iteratively distilled into larger, more robust policies using a combination of reinforcement learning and supervised learning. Our final SuperPADL controller is trained on a dataset containing over 5000 skills and runs in real time on a consumer GPU. Moreover, our policy can naturally transition between skills, allowing for users to interactively craft multi-stage animations. We experimentally demonstrate that SuperPADL significantly outperforms RL-based baselines at this large data scale.