Jiawei He

Jiawei He

Auto-generated profile, based on publicly available information

Jiawei He

California Institute of Technology University University University of California Irvine

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 October 2018
Verified
Book Chapter
Open access
PDF file
JHJiawei He
Jiawei He
ALAndreas Lehrmann
Andreas Lehrmann
JMJoseph Marino
Joseph Marino
GMGreg Mori
Greg Mori
LSLeonid Sigal
Leonid Sigal
Videos express highly structured spatio-temporal patterns of visual data. A video can be thought of as being governed by two factors: (i) temporally invariant (e.g., person identity), or slowly varying (e.g., activity), attribute-induced appearance, encoding the persistent content of each frame, and (ii) an inter-frame motion or scene dynamics (e.g., encoding evolution of the person executing the action). Based on this intuition, we propose a generative framework for video generation and future prediction. The proposed framework generates a video (short clip) by decoding samples sequentially drawn from a latent space distribution into full video frames. Variational Autoencoders (VAEs) are used as a means of encoding/decoding frames into/from the latent space and RNN as a way to model the dynamics in the latent space. We improve the video generation consistency through temporally-conditional sampling and quality by structuring the latent space with attribute controls; ensuring that attributes can be both inferred and conditioned on during learning/generation. As a result, given attributes and/or the first frame, our model is able to generate diverse but highly consistent sets of video sequences, accounting for the inherent uncertainty in the prediction task. Experimental results on three challenging datasets, along with detailed comparison to the state-of-the-art, verify effectiveness of the framework.

18 November 2021 in Machine Learning
Verified
Journal Article
Closed access
PDF file
JMJoseph Marino
Joseph Marino
LCLei Chen
Lei Chen
JHJiawei He
Jiawei He
SMStephan Mandt
Stephan Mandt

26 July 2021
Verified
Conference Paper
Closed access
XCXingyu Cui
Xingyu Cui
Huifeng ZhangHuifeng Zhang
Huifeng Zhang
JHJiawei He
Jiawei He
In order to solve the difficulties of multiobjective optimization problems (MOPs) such as high dimensionality and complex calculation, this paper designs a multiobjective optimization algorithm of multi-subpopulation coevolution. In this paper, the MOPs is divided into several sub-problems, and multiple sub-populations are used to optimize the sub-problems separately, and an information exchange strategy is designed for coevolution among multiple subpopulations; Aiming at the problem of multiobjective particle swarm optimization easily falling into local optima, an adaptive Cauchy mutation is designed to enhance the diversity and convergence of the algorithm.

8 March 2022 in arXiv
Verified
Preprint
Open access
PDF file
JMJoseph Marino
Joseph Marino
LCLei Chen
Lei Chen
JHJiawei He
Jiawei He
SMStephan Mandt
Stephan Mandt
We propose an approach for improving sequence modeling based on autoregressive normalizing flows. Each autoregressive transform, acting across time, serves as a moving frame of reference, removing temporal correlations, and simplifying the modeling of higher-level dynamics. This technique provides a simple, general-purpose method for improving sequence modeling, with connections to existing and classical techniques. We demonstrate the proposed approach both with standalone flow-based models and as a component within sequential latent variable models. Results are presented on three benchmark video datasets, where autoregressive flow-based dynamics improve log-likelihood performance over baseline models. Finally, we illustrate the decorrelation and improved generalization properties of using flow-based dynamics.

30 May 2017 in arXiv
Verified
Preprint
Open access
PDF file
JHJiawei He
Jiawei He
MIMostafa S. Ibrahim
Mostafa S. Ibrahim
ZDZhiwei Deng
Zhiwei Deng
GMGreg Mori
Greg Mori
We develop a novel framework for action localization in videos. We propose the Tube Proposal Network (TPN), which can generate generic, class-independent, video-level tubelet proposals in videos. The generated tubelet proposals can be utilized in various video analysis tasks, including recognizing and localizing actions in videos. In particular, we integrate these generic tubelet proposals into a unified temporal deep network for action classification. Compared with other methods, our generic tubelet proposal method is accurate, general, and is fully differentiable under a smoothL1 loss function. We demonstrate the performance of our algorithm on the standard UCF-Sports, J-HMDB21, and UCF-101 datasets. Our class-independent TPN outperforms other tubelet generation methods, and our unified temporal deep network achieves state-of-the-art localization results on all three datasets.