Junpeng Liu Liu

Mr. Junpeng Liu Liu

Beijing University of Posts and Telecommunications, Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 March 2022 in Applied Sciences
Verified
Journal Article
Open access
PDF file
ZBZiwei Bai
Ziwei Bai
JLJunpeng Liu Liu
Junpeng Liu Liu
MWMeiqi Wang
Meiqi Wang
CYCaixia Yuan
Caixia Yuan
XWXiaojie Wang
Xiaojie Wang
Answering different multi-choice machine reading comprehension (MRC) questions generally requires different information due to the abundant diversity of the questions, options and passages. Recently, pre-trained language models which provide rich information have been widely used to address MRC tasks. Most of the existing work only focuses on the output representation at the top layer of the models; the subtle and beneficial information provided by the intermediate layers is ignored. This paper therefore proposes a multi-decision based transformer model that builds multiple decision modules by utilizing the outputs at different layers to confront the various questions and passages. To avoid the information diversity in different layers being damaged during fine-tuning, we also propose a learning rate decaying method to control the updating speed of the parameters in different blocks. Experimental results on multiple publicly available datasets show that our model can answer different questions by utilizing the representation in different layers and speed up the inference procedure with considerable accuracy.