Yachao Li

Yachao Li

Auto-generated profile, based on publicly available information

Yachao Li

Key Laboratory of Linguistic and Cultural Computing, Ministry of Education, Northwest Minzu University, Lanzhou 730030, China

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
19 September 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Closed access
YLYachao Li
Yachao Li
JLJunhui Li
Junhui Li
JJJing Jiang
Jing Jiang
STShimin Tao
Shimin Tao
HYHao Yang
Hao Yang
MZMin Zhang
Min Zhang
Directly training a document-to-document (Doc2Doc) neural machine translation (NMT) via Transformer from scratch, especially on small datasets, usually fails to converge. Our dedicated probing tasks show that 1) both the absolute position and relative position information gets gradually weakened or even vanished once it reaches the upper encoder layers, and 2) the vanishing of absolute position information in encoder output causes the training failure of Doc2Doc NMT. To alleviate this problem, we propose a position-aware Transformer (P-Transformer) to enhance both the absolute and relative position information in both self-attention and cross-attention. Specifically, we integrate absolute positional information, i.e., position embeddings, into the query-key pairs both in self-attention and cross-attention through a simple yet effective addition operation. Moreover, we also integrate relative position encoding in self-attention. The proposed P-Transformer utilizes sinusoidal position encoding and does not require any task-specified position embedding, segment embedding, or attention mechanism. Through the above methods, we build a Doc2Doc NMT model with P-Transformer, which ingests the source document and completely generates the target document in a sequence-to-sequence (seq2seq) way. In addition, P-Transformer can be applied to seq2seq-based document-to-sentence (Doc2Sent) and sentence-to-sentence (Sent2Sent) translations. Extensive experimental results of Doc2Doc NMT show that P-Transformer significantly outperforms strong baselines on the widely-used 9 document-level datasets in 7 language pairs, covering small-, middle-, and large-scales, and achieves a new state-of-the-art. Experimentation on discourse phenomena shows that our Doc2Doc NMT models improve the translation quality in both BLEU and discourse coherence. We make our code available on Github.

5 January 2024 in Journal of Molecular Graphics and Modelling
Verified
Journal Article
Closed access
JJJing Jiang
Jing Jiang
YLYachao Li
Yachao Li
RZRuisheng Zhang
Ruisheng Zhang
YLYunwu Liu
Yunwu Liu

13 November 2025 in Neurocomputing
Verified
Journal Article
Closed access
YLYachao Li
Yachao Li
JLJunhui Li
Junhui Li
JJJing Jiang
Jing Jiang
MZMin Zhang
Min Zhang

1 November 2020 in Journal of Physics: Conference Series
Verified
Conference Paper
Open access
PDF file
DLDing Liu
Ding Liu
YLYachao Li
Yachao Li
DZDengyun Zhu
Dengyun Zhu
XLXuan Liu
Xuan Liu
NMNing Ma
Ning Ma
AZAo Zhu
Ao Zhu
In recent years, the proposal of neural network has provided new idea for solving natural language processing, and at the same time, neural machine translation has become the frontier method of machine translation. In low-resource languages, due to the sparse bilingual data, the model needs more high-quality data, and the translation quality fails to achieve the desired effect. In this paper, experiments on neural network machine translation based on attention are conducted on Tibetan-Chinese language pairs, and transfer learning method combined with back translation method is used to alleviate the problem of insufficient Tibetan-Chinese parallel corpus. Experimental results show that the proposed transfer learning combined with back translation method is simple and effective. Compared with traditional translation methods, the translation effect is significantly improved. From the analysis of translation, it can be seen that the citation of Tibetan-Chinese neural machine translation is smoother, which is greatly improved compared to the translation without back translation. At the same time, there are common deficiencies in neural machine translation such as inadequate translation and low translation loyalty.

6 May 2022
Verified
Conference Paper
Closed access
PDF file
YWYing Wang
Ying Wang
ZWZehao Wang
Zehao Wang
HLHong Li
Hong Li
YLYachao Li
Yachao Li
FZFang Zuo
Fang Zuo
In order to facilitate government departments to assess security risks and prevent infiltration, it’s necessary to recognize the IoT device from open data by the method of Named entity recognition (NER). In this study, a hybrid method of chinese NER which contains neural networks and templates is proposed to extract IoT device information from open web. The model first initiates characters of input through BERT, which can create character embedding while reduce dependency with external datasets. Then, the initiated vectors are fed into a multi-layers BiLSTM model to encode the contextual representation of each character, and a linear CRF layer after BiLSTM is used to assign the scores to every character for entity annotation. In addition, a dictionary and rule base is constructed based on the characteristics of IoT devices to correct the annotation results. We experimented the methods on the dataset of IoT, the results prove the proposed model achieves better recognition effect than other models, and the F-scores value is 88.9%.