Wei Zhang, Hongcheng Guo, Jian Yang, Zhoujin Tian, Yi Zhang, Yan Chaoran, Zhoujun Li, Tongliang Li, Xu Shi, Liangfan Zheng, Bo Zhang. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

Jian Yang, Shaohan Huang, Shuming Ma, Yuwei Yin, Li Dong, Dongdong Zhang, Hongcheng Guo, Zhoujun Li, Furu Wei. Findings of the Association for Computational Linguistics: EMNLP 2022. 2022.

Tao Sun, Linzheng Chai, Jian Yang, Yuwei Yin, Hongcheng Guo, Jiaheng Liu, Bing Wang, Liqun Yang, Zhoujun Li. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

Knowledge-grounded conversation has led to great progress in producing informative dialog responses by leveraging external knowledge. This work focuses on two affiliated knowledge grounded conversation tasks: Knowledge Selection and Response Generation . Previous work followed the paradigm of selecting the most optimal knowledge piece to guide the conversation towards generating the proper response. However, some knowledge pieces, which are not recognized as optimal, may still benefit response generation. How to effectively leverage these relevant knowledge pieces for response generation still remain a tricky issue. To address this problem, we propose KIN ET , a Knowledge Incorporation Network, which deals with the problem by boosting both the knowledge selection and the response generation. The proposed model contains a negative enhanced knowledge approximator which improves knowledge selection by enhancing the dense representation of knowledge pieces, and a curriculum knowledge sampler which improves generated responses by incorporating more relevant knowledge pieces in an easy-to-hard manner. We conduct the experiment on two datasets of knowledge-grounded conversations, the results show that the proposed model significantly outperforms state-of-the-art methods in terms of both automatic and human evaluations.

Retrieval-enhanced text generation has shown remarkable progress on knowledge-intensive language tasks, such as open-domain question answering and knowledge-enhanced dialogue generation, by leveraging passages retrieved from a large passage corpus for delivering a proper answer given the input query. However, the retrieved passages are not ideal for guiding answer generation because of the discrepancy between retrieval and generation, i.e., the candidate passages are all treated equally during the retrieval procedure without considering their potential to generate a proper answer. This discrepancy makes a passage retriever deliver a sub-optimal collection of candidate passages to generate the answer. In this paper, we propose the GeneRative Knowledge Improved Passage Ranking (GripRank) approach, addressing the above challenge by distilling knowledge from a generative passage estimator (GPE) to a passage ranker, where the GPE is a generative language model used to measure how likely the candidate passages can generate the proper answer. We realize the distillation procedure by teaching the passage ranker learning to rank the passages ordered by the GPE. Furthermore, we improve the distillation quality by devising a curriculum knowledge distillation mechanism, which allows the knowledge provided by the GPE can be progressively distilled to the ranker through an easy-to-hard curriculum, enabling the passage ranker to correctly recognize the provenance of the answer from many plausible candidates. We conduct extensive experiments on four datasets across three knowledge-intensive language tasks. Experimental results show advantages over the state-of-the-art methods for both passage ranking and answer generation on the KILT benchmark.