Various tasks are reformulated as multi-label classification problems, in which the binary cross-entropy (BCE) loss is frequently utilized for optimizing well-designed models. However, the vanilla BCE loss cannot be tailored for diverse tasks, resulting in a suboptimal performance for different models. Besides, the imbalance between redundant negative samples and rare positive samples could degrade the model performance. In this paper, we propose an effective Asymmetric Polynomial Loss (APL) to mitigate the above issues. Specifically, we first perform Taylor expansion on BCE loss. Then we ameliorate the coefficients of polynomial functions. We further employ the asymmetric focusing mechanism to decouple the gradient contribution from the negative and positive samples. Moreover, we validate that the polynomial coefficients can recalibrate the asymmetric focusing hyperparameters. Experiments on relation extraction, text classification, and image classification show that our APL loss can consistently improve performance without extra training burden. 1
Generating SPARQL queries from natural language questions is challenging in Knowledge Base Question Answering (KBQA) systems. The current state-of-the-art models heavily rely on fine-tuning pretrained models such as T5. However, these methods still encounter critical issues such as triple-flip errors (e.g., (subject, relation, object) is predicted as (object, relation, subject)). To address this limitation, we introduce TSET (Triplet Structure Enhanced T5), a model with a novel pretraining stage positioned between the initial T5 pretraining and the fine-tuning for the Text-to-SPARQL task. In this intermediary stage, we introduce a new objective called Triplet Structure Correction (TSC) to train the model on a SPARQL corpus derived from Wikidata. This objective aims to deepen the model’s understanding of the order of triplets. After this specialized pretraining, the model undergoes fine-tuning for SPARQL query generation, augmenting its query-generation capabilities. We also propose a method named “semantic transformation” to fortify the model’s grasp of SPARQL syntax and semantics without compromising the pre-trained weights of T5. Experimental results demonstrate that our proposed TSET outperforms existing methods on three well-established KBQA datasets: LC-QuAD 2.0, QALD-9 plus, and QALD-10, establishing a new state-of-the-art performance (95.0% F1 and 93.1% QM on LC-QuAD 2.0, 75.85% F1 and 61.76% QM on QALD-9 plus, 51.37% F1 and 40.05% QM on QALD-10).
24 July 2024 in Journal of Data and Information Science
Purpose: This article presents an in-depth analysis of global research trends in Geosciences from 2014 to 2023. By integrating bibliometric analysis with expert insights from the Deeptime Digital Earth (DDE) initiative, this article identifies key emerging themes shaping the landscape of Earth Sciences ① . Design/methodology/approach: The identification process involved a meticulous analysis of over 400,000 papers from 466 Geosciences journals and approximately 5,800 papers from 93 interdisciplinary journals sourced from the Web of Science and Dimensions database. To map relationships between articles, citation networks were constructed, and spectral clustering algorithms were then employed to identify groups of related research, resulting in 407 clusters. Relevant research terms were extracted using the Log-Likelihood Ratio (LLR) algorithm, followed by statistical analyses on the volume of papers, average publication year, and average citation count within each cluster. Additionally, expert knowledge from DDE Scientific Committee was utilized to select top 30 trends based on their representation, relevance, and impact within Geosciences, and finalize naming of these top trends with consideration of the content and implications of the associated research. This comprehensive approach in systematically delineating and characterizing the trends in a way which is understandable to geoscientists. Findings: Thirty significant trends were identified in the field of Geosciences, spanning five domains: deep space, deep time, deep Earth, habitable Earth, and big data. These topics reflect the latest trends and advancements in Geosciences and have the potential to address real-world problems that are closely related to society, science, and technology. Research limitations: The analyzed data of this study only contain those were included in the Web of Science. Practical implications: This study will strongly support the organizations and individual scientists to understand the modern frontier of earth science, especially on solid earth. The organizations such as the surveys or natural science fund could map out areas for future exploration and analyze the hot topics reference to this study. Originality/value: This paper integrates bibliometric analysis with expert insights to highlight the most significant trends on earth science and reach the individual scientist and public by global voting.
Yi Xu, Shuqian Sheng, Jiexing Qi, Luoyi Fu, Zhouhan Lin, Xinbing Wang, Chenghu Zhou. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.
18 July 2024 in IEEE Transactions on Audio, Speech and Language Processing
Pretrained language models (PLMs) have made remarkable progress in table-to-text generation tasks. However, the lack of domain-specific knowledge makes it challenging to bridge the topological gap between tabular data and text, especially in real-world applications with limited resources. To mitigate the limitation of insufficient labeled data, we propose a novel framework: Adapt-Knowledge-to-Generate (AKG). The core insight of AKG is to adapt unlabeled domain-specific knowledge into the model, which brings at least three benefits: (1) it injects representation of normal table-related descriptions to bridge the topological gap between tabular data and texts; (2) it enables us to use large amounts of unlabeled domain-specific knowledge fully, which can alleviate the PLMs' inherent shortcomings of lacking domain knowledge; (3) it allows us to design various tasks to employ the domain-specific knowledge. Extensive experiments and analyses are conducted on three open-domain, few-shot natural language generation (NLG) data sets: Humans, Songs, and Books. Compared to previous state-of-the-art approaches, our model achieves superior performance in terms of both fluency and accuracy as judged by human and automatic evaluations.