Recent years have witnessed a substantial increase in the use of deep learning to solve various natural language processing (NLP) problems. Early deep learning models were constrained by their sequential or unidirectional nature, such that they struggled to capture the contextual relationships across text inputs. The introduction of bidirectional encoder representations from transformers (BERT) leads to a robust encoder for the transformer model that can understand the broader context and deliver state-of-the-art performance across various NLP tasks. This has inspired researchers and practitioners to apply BERT to practical problems, such as information retrieval (IR). A survey that focuses on a comprehensive analysis of prevalent approaches that apply pretrained transformer encoders like BERT to IR can thus be useful for academia and the industry. In light of this, we revisit a variety of BERT-based methods in this survey, cover a wide range of techniques of IR, and group them into six high-level categories: (i) handling long documents, (ii) integrating semantic information, (iii) balancing effectiveness and efficiency, (iv) predicting the weights of terms, (v) query expansion, and (vi) document expansion. We also provide links to resources, including datasets and toolkits, for BERT-based IR systems. Additionally, we highlight the advantages of employing encoder-based BERT models in contrast to recent large language models like ChatGPT, which are decoder-based and demand extensive computational resources. Finally, we summarize the comprehensive outcomes of the survey and suggest directions for future research in the area.
Drug-drug interaction (DDI) pertains to the occurrence where the concomitant use of two or more drugs may lead to interactions in terms of their pharmacokinetic or pharmacodynamic behavior, resulting in unexpected effects. Accurately predicting DDIs holds significant importance in ensuring drug safety. Despite the numerous approaches proposed for DDI prediction, a majority of these methods often overlook the challenge presented by cold-start scenario, consequently limiting their applicability. This paper presents a novel data augmentation approach for the prediction of DDIs in cold-start scenarios. This method leverages counterfactual inference to generate meaningful pseudo samples for drugs with limited prior information. To achieve this, a HIN relevant to DDIs is initially established by amalgamating various associations between drugs and proteins. Subsequently, the identification of drug communities within this HIN is regarded as a form of counterfactual inference treatment, facilitating the generation of counterfactual links for cold-start drugs and thereby augmenting the training dataset. Lastly, we enhance our understanding of drug characteristics through a meta-path-based fusion mechanism, ultimately improving the accuracy of DDIs prediction in cold-start scenarios. We substantiate the effectiveness of our proposed method through an extensive series of experiments.
Recent advancements in large language models have led to significant improvements in various natural language processing tasks, including automated question answering. However, these models still struggle with providing accurate responses to complex biology questions, such as those found in the National College Entrance Examination. To address this issue, researchers have explored retrieval-augmented generation techniques, which incorporate external knowledge sources to enhance answer accuracy. While these methods have shown promise in improving efficiency, they often lack the ability to perform in-depth analysis of complex questions. In response to these limitations, the paper proposes a novel ReAct agent-based question answering method. This method combines biology textbook document libraries with Google search engine capabilities, leveraging the strengths of large language models. By integrating these components, this method can engage in autonomous decision-making and multi-step reasoning, allowing for a more thorough analysis of complex biology questions. Experimental results demonstrate the effectiveness of this new approach. When compared to responses generated directly by ChatGPT and GPT-4o models, the method showed significant improvements in accuracy for biology questions from the National College Entrance Examination. Specifically, the new method achieved accuracy increases of 11.1% and 4.64% over ChatGPT and GPT-4o, respectively.