Zhipeng Fan

Zhipeng Fan

Auto-generated profile, based on publicly available information

Zhipeng Fan

New York University

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 March 2023 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
JPJianhong Pan
Jianhong Pan
LFLin Geng Foo
Lin Geng Foo
QZQichen Zheng
Qichen Zheng
ZFZhipeng Fan
Zhipeng Fan
Hossein RahmaniHossein Rahmani
Hossein Rahmani
JLJared Liu
Jared Liu
Dynamic neural networks can greatly reduce computation redundancy without compromising accuracy by adapting their structures based on the input. In this paper, we explore the robustness of dynamic neural networks against energy-oriented attacks targeted at reducing their efficiency. Specifically, we attack dynamic models with our novel algorithm GradMDM. GradMDM is a technique that adjusts the direction and the magnitude of the gradients to effectively find a small perturbation for each input, that will activate more computational units of dynamic models during inference. We evaluate GradMDM on multiple datasets and dynamic models, where it outperforms previous energy-oriented attack techniques, significantly increasing computation complexity while reducing the perceptibility of the perturbations https://github.com/lingengfoo/GradMDM .

28 October 2022
Verified
Book Chapter
Closed access
PDF file
JPJianhong Pan
Jianhong Pan
QZQichen Zheng
Qichen Zheng
ZFZhipeng Fan
Zhipeng Fan
Hossein RahmaniHossein Rahmani
Hossein Rahmani
QKQiuhong Ke
Qiuhong Ke
JLJared Liu
Jared Liu

30 June 2023 in IEEE Transactions on Multimedia
Verified
Journal Article
Closed access
JPJianhong Pan
Jianhong Pan
SYSiyuan Yang
Siyuan Yang
LFLin Geng Foo
Lin Geng Foo
QKQiuhong Ke
Qiuhong Ke
Hossein RahmaniHossein Rahmani
Hossein Rahmani
JLJared Liu
Jared Liu
Currently, salience-based channel pruning makes continuous breakthroughs in network compression. In the realization, the salience mechanism is used as a metric of channel salience to guide pruning. Therefore, salience-based channel pruning can dynamically adjust the channel width at run-time, which provides a flexible pruning scheme. However, there are two problems emerging: a gating function is often needed to truncate the specific salience entries to zero, which destabilizes the forward propagation; dynamic architecture brings more cost for indexing in inference which bottlenecks the inference speed. In this paper, we propose a Progressive Channel-Shrinking (PCS) method to compress the selected salience entries at run-time instead of roughly approximating them to zero. We also propose a Running Shrinking Policy to provide a testing-static pruning scheme that can reduce the memory access cost for filter indexing. We evaluate our method on ImageNet and CIFAR10 datasets over two prevalent networks: ResNet and VGG, and demonstrate that our PCS outperforms all baselines and achieves state-of-the-art in terms of compression-performance tradeoff. Moreover, we observe a significant and practical acceleration of inference. The code will be released upon acceptance.

9 April 2023 in arXiv
Verified
Preprint
Open access
PDF file
JGJia Gong
Jia Gong
LFLin Geng Foo
Lin Geng Foo
ZFZhipeng Fan
Zhipeng Fan
QKQiuhong Ke
Qiuhong Ke
Hossein RahmaniHossein Rahmani
Hossein Rahmani
JLJun Liu
Jun Liu
Monocular 3D human pose estimation is quite challenging due to the inherent ambiguity and occlusion, which often lead to high uncertainty and indeterminacy. On the other hand, diffusion models have recently emerged as an effective tool for generating high-quality images from noise. Inspired by their capability, we explore a novel pose estimation framework (DiffPose) that formulates 3D pose estimation as a reverse diffusion process. We incorporate novel designs into our DiffPose to facilitate the diffusion process for 3D pose estimation: a pose-specific initialization of pose uncertainty distributions, a Gaussian Mixture Model-based forward diffusion process, and a context-conditioned reverse diffusion process. Our proposed DiffPose significantly outperforms existing methods on the widely used pose estimation benchmarks Human3.6M and MPI-INF-3DHP. Project page: https://gongjia0208.github.io/Diffpose/.


9 April 2023 in arXiv
Verified
Preprint
Open access
PDF file
LFLin Geng Foo
Lin Geng Foo
JGJia Gong
Jia Gong
ZFZhipeng Fan
Zhipeng Fan
JLJun Liu
Jun Liu
Recent years have witnessed great progress in deep neural networks for real-time applications. However, most existing works do not explicitly consider the general case where the device's state and the available resources fluctuate over time, and none of them investigate or address the impact of varying computational resources for online video understanding tasks. This paper proposes a System-status-aware Adaptive Network (SAN) that considers the device's real-time state to provide high-quality predictions with low delay. Usage of our agent's policy improves efficiency and robustness to fluctuations of the system status. On two widely used video understanding tasks, SAN obtains state-of-the-art performance while constantly keeping processing delays low. Moreover, training such an agent on various types of hardware configurations is not easy as the labeled training data might not be available, or can be computationally prohibitive. To address this challenging problem, we propose a Meta Self-supervised Adaptation (MSA) method that adapts the agent's policy to new hardware configurations at test-time, allowing for easy deployment of the model onto other unseen hardware platforms.