Fu Wang

Mr. Fu Wang


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 October 2021 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Verified
Journal Article
Open access
LXLiao Xianwen
Liao Xianwen
HYHuang Yongzhong
Huang Yongzhong
YWYongzhuang Wei
Yongzhuang Wei
CZChenhao Zhang
Chenhao Zhang
FWFu Wang
Fu Wang
Yong WangYong Wang
Yong Wang
Sentence representation is an important research hotspot in natural language processing (NLP) since it can map the semantics of sentences into semantics vectors, thereby effectively solving complex semantics computing problems. Recently, sentence representations are mainly obtained by indirect means. Specifically, for sentence representations obtained by unsupervised means, they are often calculated by the weighted sum of embeddings of tokens in sentences; for sentence representations obtained by self-supervised or supervised means, they are often derived from intermediate encodings of sentences in prediction tasks. For example, Google's BERT and MUSE respectively use the embedding of [CLS] in the next sentence prediction task and intermediate encodings of sentences in the translation bridge task as sentence representations. In this paper, we use the observed semantics increment feature of sentences to directly model the semantics function of sentences. To be able to use the existing neural network language model to approximate the semantics function, we first implement the first-order Taylor expansion on the semantics function to obtain a difference semantics model and then add it to BERT as a subtask to perform self-supervised fine-tuning. Finally, we get a new sentence representation model S-BERT. S-BERT achieves the state-of-the-art performance on many datasets in Chinese, English, and Vietnamese.