Lianyu Hu

Lianyu Hu

Auto-generated profile, based on publicly available information

Lianyu Hu

Nanyang technological university, Singapore, Singapore

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 November 2022
Verified
Book Chapter
Closed access
PDF file
LHLianyu Hu
Lianyu Hu
LGLiqing Gao
Liqing Gao
ZLZekang Liu
Zekang Liu
FFFeng Feng
Feng Feng

26 October 2023
Verified
Conference Paper
Open access
LHLianyu Hu
Lianyu Hu
LGLiqing Gao
Liqing Gao
ZLZekang Liu
Zekang Liu
Chi-Man PunChi-Man Pun
Chi-Man Pun
FFFeng Feng
Feng Feng
Raw videos have been proven to own considerable feature redundancy where in many cases only a portion of frames can already meet the requirements for accurate recognition. In this paper, we are interested in whether such redundancy can be effectively leveraged to facilitate efficient inference in continuous sign language recognition (CSLR). We propose a novel adaptive model (AdaBrowse) to dynamically select a most informative subsequence from input video sequences by modelling this problem as a sequential decision task. In specific, we first utilize a lightweight network to quickly scan input videos to extract coarse features. Then these features are fed into a policy network to intelligently select a subsequence to process. The corresponding subsequence is finally inferred by a normal CSLR model for sentence prediction. As only a portion of frames are processed in this procedure, the total computations can be considerably saved. Besides temporal redundancy, we are also interested in whether the inherent spatial redundancy can be seamlessly integrated together to achieve further efficiency, i.e., dynamically selecting a lowest input resolution for each sample, whose model is referred to as AdaBrowse+. Extensive experimental results on four large-scale CSLR datasets, i.e., PHOENIX14, PHOENIX14-T, CSL-Daily and CSL, demonstrate the effectiveness of AdaBrowse and AdaBrowse+ by achieving comparable accuracy with state-of-the-art methods with 1.44X throughput and 2.12X fewer FLOPs. Comparisons with other commonly-used 2D CNNs and adaptive efficient methods verify the effectiveness of AdaBrowse. Code is available at https://github.com/hulianyuyy/AdaBrowse.

26 June 2023 in Expert Systems with Applications
Verified
Journal Article
Closed access
LHLianyu Hu
Lianyu Hu
SLShenglan Liu
Shenglan Liu
FFFeng Feng
Feng Feng

29 October 2024
Verified
Book Chapter
Closed access
PDF file
TSTongkai Shi
Tongkai Shi
LHLianyu Hu
Lianyu Hu
Fanhua ShangFanhua Shang
Fanhua Shang
JFJichao Feng
Jichao Feng
PLPeidong Liu
Peidong Liu
FFFeng Feng
Feng Feng

30 July 2023 in The Visual Computer
Verified
Journal Article
Closed access
PDF file
LGLiqing Gao
Liqing Gao
LHLianyu Hu
Lianyu Hu
FLFan Lyu
Fan Lyu
LZLei Zhu
Lei Zhu
LWLiang Wan
Liang Wan
FFFeng Feng
Feng Feng

2 April 2024 in IEEE Transactions on Emerging Topics in Computational Intelligence
Verified
Journal Article
Closed access
LHLianyu Hu
Lianyu Hu
LGLiqing Gao
Liqing Gao
ZLZekang Liu
Zekang Liu
FFFeng Feng
Feng Feng
Despite the recent progress of continuous sign language recognition (CSLR), most state-of-the-art methods process input sign language videos frame by frame to predict sentences. This usually causes a heavy computational burden and is inefficient and even infeasible in real-world scenarios. Inspired by the fact that videos are inherently redundant where not all frames are essential for recognition, we propose spatial temporal aggregation (STAgg) to address this problem. Specifically, STAgg synthesizes adjacent similar frames into a unified robust representation before being fed into the recognition module, thus highly reducing the computation complexity and memory demand. We first give a detailed analysis on commonly-used aggregation methods like subsampling, max pooling and average, and then naturally derive our STAgg from the expected design criterion. Compared to commonly used pooling and subsampling counterparts, extensive ablation studies verify the superiority of our proposed three diverse STAgg variants in both accuracy and efficiency. The best version achieves comparative accuracy with state-of-the-art competitors, but is 1.35× faster with only 0.50× computational costs, consuming 0.70× training time and 0.65× memory usage. Experiments on four large-scale datasets upon multiple backbones fully verify the generalizability and effectiveness of the proposed STAgg. Another advantage of STAgg is enabling more powerful backbones, which may further boost the accuracy of CSLR under similar computational/memory budgets. We also visualize the results of STAgg to support intuitive and insightful analysis of the effects of STAgg.