Zhe Wang

Zhe Wang

Auto-generated profile, based on publicly available information

Zhe Wang

Deying Kong University of California Irvine

Research Focus & Metrics

Academic Career Length

12Years (2015-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 September 2016
Verified
Book Chapter
Closed access
PDF file
LWLimin Wang
Limin Wang
YXYuanjun Xiong
Yuanjun Xiong
ZWZhe Wang
Zhe Wang
YQYu Qiao
Yu Qiao
DLDahua Lin
Dahua Lin
LGLuc Van Gool
Luc Van Gool

3 September 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
LWLimin Wang
Limin Wang
YXYuanjun Xiong
Yuanjun Xiong
ZWZhe Wang
Zhe Wang
YQYu Qiao
Yu Qiao
DLDahua Lin
Dahua Lin
LGLuc Van Gool
Luc Van Gool
Deep convolutional networks have achieved great success for image recognition. However, for action recognition in videos, their advantage over traditional methods is not so evident. We present a general and flexible video-level framework for learning action models in videos. This method, called temporal segment network (TSN), aims to model long-range temporal structures with a new segment-based sampling and aggregation module. This unique design enables our TSN to efficiently learn action models by using the whole action videos. The learned models could be easily adapted for action recognition in both trimmed and untrimmed videos with simple average pooling and multi-scale temporal window integration, respectively. We also study a series of good practices for the instantiation of TSN framework given limited training samples. Our approach obtains the state-the-of-art performance on four challenging action recognition benchmarks: HMDB51 (71.0%), UCF101 (94.9%), THUMOS14 (80.1%), and ActivityNet v1.2 (89.6%). Using the proposed RGB difference for motion models, our method can still achieve competitive accuracy on UCF101 (91.0 %) while running at 340 FPS. Furthermore, based on the temporal segment networks, we won the video classification track at the ActivityNet challenge 2016 among 24 teams, which demonstrates the effectiveness of TSN and the proposed good practices.

1 June 2016
Verified
Conference Paper
Closed access
BZBowen Zhang
Bowen Zhang
LWLimin Wang
Limin Wang
ZWZhe Wang
Zhe Wang
YQYu Qiao
Yu Qiao
HWHanli Wang
Hanli Wang
The deep two-stream architecture [23] exhibited excellent performance on video based action recognition. The most computationally expensive step in this approach comes from the calculation of optical flow which prevents it to be real-time. This paper accelerates this architecture by replacing optical flow with motion vector which can be obtained directly from compressed videos without extra calculation. However, motion vector lacks fine structures, and contains noisy and inaccurate motion patterns, leading to the evident degradation of recognition performance. Our key insight for relieving this problem is that optical flow and motion vector are inherent correlated. Transferring the knowledge learned with optical flow CNN to motion vector CNN can significantly boost the performance of the latter. Specifically, we introduce three strategies for this, initialization transfer, supervision transfer and their combination. Experimental results show that our method achieves comparable recognition performance to the state-of-the-art, while our method can process 390.7 frames per second, which is 27 times faster than the original two-stream method.

8 January 2018 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
BZBowen Zhang
Bowen Zhang
LWLimin Wang
Limin Wang
ZWZhe Wang
Zhe Wang
YQYu Qiao
Yu Qiao
HWHanli Wang
Hanli Wang
The two-stream CNNs prove very successful for video-based action recognition. However, the classical two-stream CNNs are time costly, mainly due to the bottleneck of calculating optical flows (OFs). In this paper, we propose a two-stream-based real-time action recognition approach by using motion vector (MV) to replace OF. MVs are encoded in video stream and can be extracted directly without extra calculation. However, directly training CNN with MVs degrades accuracy severely due to the noise and the lack of fine details in MVs. In order to relieve this problem, we propose four training strategies which leverage the knowledge learned from OF CNN to enhance the accuracy of MV CNN. Our insight is that MV and OF share inherent similar structures which allow us to transfer knowledge from one domain to another. To fully utilize the knowledge learned in OF domain, we develop deeply transferred MV CNN. Experimental results on various datasets show the effectiveness of our training strategies. Our approach is significantly faster than OF based approaches and achieves processing speed of 390.7 frames per second, surpassing real-time requirement. We release our model and code to facilitate further research.

3 November 2022
Verified
Book Chapter
Closed access
PDF file
HMHaoyu Ma
Haoyu Ma
ZWZhe Wang
Zhe Wang
YCYifei Chen
Yifei Chen
DKDeying Kong
Deying Kong
LCLiangjian Chen
Liangjian Chen
XXXiaohui Xie
Xiaohui Xie

9 February 2017 in IEEE Transactions on Image Processing
Verified
Journal Article
Closed access
ZWZhe Wang
Zhe Wang
LWLimin Wang
Limin Wang
YWYali Wang
Yali Wang
BZBowen Zhang
Bowen Zhang
YQYu Qiao
Yu Qiao
Traditional feature encoding scheme (e.g., Fisher vector) with local descriptors (e.g., SIFT) and recent convolutional neural networks (CNNs) are two classes of successful methods for image recognition. In this paper, we propose a hybrid representation, which leverages the discriminative capacity of CNNs and the simplicity of descriptor encoding schema for image recognition, with a focus on scene recognition. To this end, we make three main contributions from the following aspects. First, we propose a patch-level and end-to-end architecture to model the appearance of local patches, called PatchNet. PatchNet is essentially a customized network trained in a weakly supervised manner, which uses the image-level supervision to guide the patch-level feature extraction. Second, we present a hybrid visual representation, called VSAD, by utilizing the robust feature representations of PatchNet to describe local patches and exploiting the semantic probabilities of PatchNet to aggregate these local patches into a global representation. Third, based on the proposed VSAD representation, we propose a new state-of-the-art scene recognition approach, which achieves an excellent performance on two standard benchmarks: MIT Indoor67 (86.2%) and SUN397 (73.0%).