Sen Yang

Sen Yang

Auto-generated profile, based on publicly available information

Sen Yang

TRIP.COM, China

Research Focus & Metrics

Representative Publications

View All Publications
23 December 2022
Verified
Conference Paper
Open access
CLChao Luo
Chao Luo
ZCZi Chen
Zi Chen
XJXiaolin Jiang
Xiaolin Jiang
SYSen Yang
Sen Yang
Recently Transformer-based pre-trained models have become a new paradigm for text summarization, and one of the milestones is PEGASUS. It suggests the pre-training objective Gap Sentences Generation designed for summarization. However, the ROUGE1-F1-based strategy in Gap Sentences Generation is unfavorable to Chinese text summarization, considering that unigram is not the basic semantic unit of Chinese in most cases. Furthermore, ROUGE1-F1 is based upon the co-occurrences of the unigrams other than distributional semantics such as word or sentence representations. Accordingly, the gap sentences are evaluated on a biased basis and thus the performance of GSG objective pre-training is hampered. In this work, we propose an improved GSG objective that upgrades the ROUGE1-F1-based strategy with TextRank. In TextRank, the texts are first segmented to extract semantically meaningful words and form distributional representations of sentences. The significance of a sentence is assessed in light of its cosine similarity with representations of the neighboring sentences in a graph. GSG with TextRank exhibits remarkable performance gains on the LCSTS dataset, and employment of R-Drop further boosts the improvements.