20 January 2023 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
Spoken language understanding (SLU) is an essential part of a task-oriented dialogue system, which mainly includes intent detection and slot filling. Some existing approaches obtain enhanced semantic representation by establishing the correlation between two tasks. However, those methods show little improvement when applied to BERT, since BERT has learned rich semantic features. In this paper, we propose a BERT-based model with the probability-aware gate mechanism, called PAGM ( P robability A ware G ated M odel). PAGM aims to learn the correlation between intent and slot from the perspective of probability distribution, which explicitly utilizes intent information to guide slot filling. Besides, in order to efficiently incorporate BERT with the probability-aware gate, we design the stacked fine-tuning strategy. This approach introduces a mid-stage before target model training, which enables BERT to get better initialization for final training. Experiments show that PAGM achieves significant improvement on two benchmark datasets, and outperforms the previous state-of-the-art results.
22 June 2022 in IEEE Transactions on Neural Networks and Learning Systems
Dialogue state tracking (DST) is a core component of task-oriented dialogue systems. Recent works focus mainly on end-to-end DST models that omit the spoken language understanding (SLU) module to directly obtain the dialogue state based on a user’s dialogue. However, the slot information detected by slot filling in SLU is closely tied to the slot–value pair that needs to be updated in DST. Efficient use of the key slot semantic knowledge obtained by slot filling contributes to improving the performance of DST. Based on this idea, we introduce slot filling as a subtask and build an end-to-end joint model to explicitly integrate the slot information detected by slot filling, which further guides DST. In this article, a novel stack-propagation framework with slot filling for multidomain DST is proposed. The stack-propagation framework is introduced to jointly model slot filling and DST. The framework directly feeds the key slot semantic knowledge detected by slot filling into the DST module. In addition, a slot-masked attention mechanism is designed to enable DST to focus on the key slot information obtained by slot filling. When the slot value is updated, a slot–value softcopy mechanism is designed to enhance the influence of the words marked by key slots. Experiments show that our approach outperforms previous methods and performs outstandingly on two benchmark datasets.