Bridge inspection reports are an important data source in the bridge management process, and they contain a large amount of fine-grained information. However, the research on machine reading comprehension (MRC) methods for this field is insufficient, and annotating large scale domain-specific corpus is time-consuming. This paper presented a novel prompt-based few-shot MRC approach for intelligent bridge management. The proposed model uses the pretrained model MacBERT as backbone. The prompt templates are designed based on some domain-specific heuristic rules. The experimental results show that our model outperforms the baseline models in different few-shot settings. The proposed model can provide technical support for the construction of automatic question answering system in the field of bridge management.
As a crucial task in natural language processing, table question answering has garnered significant attention from both the academic and industrial communities. It enables intelligent querying and question answering over structured data by translating natural language into corresponding SQL statements. Recently, there have been notable advancements in the general domain table question answering task, achieved through prompt learning with large language models. However, in specific domains, where tables often have a higher number of columns and questions tend to be more complex, large language models are prone to generating invalid SQL or NoSQL statements. To address the above issue, this paper proposes a novel few-shot table prompt question answering approach. Specifically, we design a prompt template construction strategy for structured SQL generation. It utilizes prompt templates to restructure the input for each test data and standardizes the model output, which can enhance the integrity and validity of generated SQL. Furthermore, this paper introduces a contrastive exemplar selection approach based on the question patterns and formats in domain-specific contexts. This enables the model to quickly retrieve the relevant exemplars and learn characteristics about given question. Experimental results on the two datasets in the domains of electric energy and structural inspection show that the proposed approach outperforms the baseline models across all comparison settings.
1 January 2023 in International Journal of Intelligent Systems
In order to overcome the problems that the feature representation and classification effect of the existing methods need to be improved in complex contexts, this paper presents a novel few‐shot relation extraction approach via the entity feature enhancement and attention‐based prototypical network. The proposed model uses the pretrained RoBERTa model as the encoder while using the BiLSTM module for directional feature extraction. We further incorporate the entity feature enhancement module to improve the feature representation ability of the model. At last, the attention‐based prototypical network is used to predict relations. The experimental results show that the proposed method not only outperforms the baseline models on the datasets from the bridge inspection and health domains but also achieves competitive results on the FewRel dataset in the general domain.
31 October 2024 in IEEE/ACM Transactions on Audio, Speech, and Language Processing
The rapid development of pre-trained language models (PLMs) has significantly enhanced the performance of machine reading comprehension (MRC). Nevertheless, the traditional fine-tuning approaches necessitate extensive labeled data. MRC remains a challenging task in the few-shot settings or low-resource scenarios. This study proposes a novel few-shot MRC approach via post-training and answer span-oriented contrastive learning, termed MRC-PASCL. Specifically, in the post-training module, a novel noun-entity-aware data selection and generation strategy is proposed according to characteristics of MRC task and data, focusing on masking nouns and named entities in the context. In terms of fine-tuning, the proposed answer span-oriented contrastive learning manner selects spans around the golden answers as negative examples, and performs multi-task learning together with the standard MRC answer prediction task. Experimental results show that MRC-PASCL outperforms the PLMs-based baseline models and the 7B and 13B large language models (LLMs) cross most MRQA 2019 datasets. Further analyses show that our approach achieves better inference efficiency with lower computational resource requirement. The analysis results also indicate that the proposed method can better adapt to the domain-specific scenarios.