Pengyuan Lyu

Pengyuan Lyu

Auto-generated profile, based on publicly available information

Pengyuan Lyu

Baidu Inc

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
25 June 2018 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
BSBaoguang Shi
Baoguang Shi
MYMingkun Yang
Mingkun Yang
Xinggang WangXinggang Wang
Xinggang Wang
PLPengyuan Lyu
Pengyuan Lyu
CYCong Yao
Cong Yao
XBXiang Bai
Xiang Bai
A challenging aspect of scene text recognition is to handle text with distortions or irregular layout. In particular, perspective text and curved text are common in natural scenes and are difficult to recognize. In this work, we introduce ASTER, an end-to-end neural network model that comprises a rectification network and a recognition network. The rectification network adaptively transforms an input image into a new one, rectifying the text in it. It is powered by a flexible Thin-Plate Spline transformation which handles a variety of text irregularities and is trained without human annotations. The recognition network is an attentional sequence-to-sequence model that predicts a character sequence directly from the rectified image. The whole model is trained end to end, requiring only images and their groundtruth text. Through extensive experiments, we verify the effectiveness of the rectification and demonstrate the state-of-the-art recognition performance of ASTER. Furthermore, we demonstrate that ASTER is a powerful component in end-to-end recognition systems, for its ability to enhance the detector.

9 October 2018
Verified
Book Chapter
Closed access
PDF file
PLPengyuan Lyu
Pengyuan Lyu
MLMinghui Liao
Minghui Liao
CYCong Yao
Cong Yao
WWWenhao Wu
Wenhao Wu
XBXiang Bai
Xiang Bai

26 August 2019 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Verified
Journal Article
Closed access
MLMinghui Liao
Minghui Liao
PLPengyuan Lyu
Pengyuan Lyu
MHMinghang He
Minghang He
CYCong Yao
Cong Yao
WWWenhao Wu
Wenhao Wu
XBXiang Bai
Xiang Bai
Unifying text detection and text recognition in an end-to-end training fashion has become a new trend for reading text in the wild, as these two tasks are highly relevant and complementary. In this paper, we investigate the problem of scene text spotting, which aims at simultaneous text detection and recognition in natural images. An end-to-end trainable neural network named as Mask TextSpotter is presented. Different from the previous text spotters that follow the pipeline consisting of a proposal generation network and a sequence-to-sequence recognition network, Mask TextSpotter enjoys a simple and smooth end-to-end learning procedure, in which both detection and recognition can be achieved directly from two-dimensional space via semantic segmentation. Further, a spatial attention module is proposed to enhance the performance and universality. Benefiting from the proposed two-dimensional representation on both detection and recognition, it easily handles text instances of irregular shapes, for instance, curved text. We evaluate it on four English datasets and one multi-language dataset, achieving consistently superior performance over state-of-the-art methods in both detection and end-to-end text recognition tasks. Moreover, we further investigate the recognition module of our method separately, which significantly outperforms state-of-the-art methods on both regular and irregular text datasets for scene text recognition.

26 October 2023
Verified
Conference Paper
Open access
XQXugong Qin
Xugong Qin
PLPengyuan Lyu
Pengyuan Lyu
CZChengquan Zhang
Chengquan Zhang
YZYu Zhou
Yu Zhou
KYKun Yao
Kun Yao
WWWeiping Wang
Weiping Wang
Due to the flexible representation of arbitrary-shaped scene text and simple pipeline, bottom-up segmentation-based methods begin to be mainstream in real-time scene text detection. Despite great progress, these methods show deficiencies in robustness and still suffer from false positives and instance adhesion. Different from existing methods which integrate multiple-granularity features or multiple outputs, we resort to the perspective of representation learning in which auxiliary tasks are utilized to enable the encoder to jointly learn robust features with the main task of per-pixel classification during optimization. For semantic representation learning, we propose global-dense semantic contrast (GDSC), in which a vector is extracted for global semantic representation, then used to perform element-wise contrast with the dense grid features. To learn instance-aware representation, we propose to combine top-down modeling (TDM) with the bottom-up framework to provide implicit instance-level clues for the encoder. With the proposed GDSC and TDM, the encoder network learns stronger representation without introducing any parameters and computations during inference. Equipped with a very light decoder, the detector can achieve more robust real-time scene text detection. Experimental results on four public datasets show that the proposed method can outperform or be comparable to the state-of-the-art on both accuracy and speed. Specifically, the proposed method achieves 87.2% F-measure with 48.2 FPS on Total-Text and 89.6% F-measure with 36.9 FPS on MSRA-TD500 on a single GeForce RTX 2080 Ti GPU.

26 October 2023
Verified
Conference Paper
Open access
PLPengyuan Lyu
Pengyuan Lyu
WMWeihong Ma
Weihong Ma
HWHongyi Wang
Hongyi Wang
YYYuechen Yu
Yuechen Yu
CZChengquan Zhang
Chengquan Zhang
JWJingdong Wang
Jingdong Wang
All tables can be represented as grids. Based on this observation, we propose GridFormer, a novel approach for interpreting unconstrained table structures by predicting the vertex and edge of a grid. First, we propose a flexible table representation in the form of an M X N grid. In this representation, the vertexes and edges of the grid store the localization and adjacency information of the table. Then, we introduce a DETR-style table structure recognizer to efficiently predict this multi-objective information of the grid in a single shot. Specifically, given a set of learned row and column queries, the recognizer directly outputs the vertexes and edges information of the corresponding rows and columns. Extensive experiments on five challenging benchmarks which include wired, wireless, multi-merge-cell, oriented, and distorted tables demonstrate the competitive performance of our model over other methods.

21 April 2024 in Pattern Recognition
Verified
Journal Article
Closed access
ZLZiyan Li
Ziyan Li
LJLianwen Jin
Lianwen Jin
CZChengquan Zhang
Chengquan Zhang
JZJiaxin Zhang
Jiaxin Zhang
ZXZecheng Xie
Zecheng Xie
KYKun Yao
Kun Yao