In this review paper, we discuss the use of BERT, one of the most well-liked deep learning-based language models. The model's operation mechanism, key areas of applicability to text analytics tasks, comparisons with related models for each activity, and a description of certain proprietary models are all covered in this work. The data from several dozen original scientific studies that have been published in the last few years and have garnered the greatest interest from the scientific community were systematized in order to prepare this review. All researchers and students who wish to learn about the most recent developments in the field of natural language text analysis may find this survey helpful. Bidirectional Encoder Representations from Transformers, or BERT for short, is the subject of a thorough investigation that we present. The area of natural language processing holds significant importance in the creation of intelligent systems. To do a variety of activities, one must comprehend the sentence's correct meaning in order to produce the desired result. Computers have a hard time understanding languages because of how context constantly changes. The main challenge in natural language processing tasks is getting computers to understand text context, and BERT is seen as a revolution in this regard. It picks up the language and its meaning in a manner that is quite similar to how the human brain processes meaning from sentences. Its capacity to identify a word from both the left and right contexts of a sentence makes it special. A new era in the perception and comprehension of natural languages has been ushered in with the development of BERT, which could help computers understand natural languages more fully. This study aims to provide readers with a deeper knowledge of the BERT language model and how it is applied to different NLP tasks.

Presently, road and traffic control construction on most university campuses cannot keep up with the growth of the universities. Campus roads are not very wide, crossings do not have lights, and there are no full-time traffic management personnel. Teachers and students are prone to forming a peak flow of people when going to and from classes. This has led to a constant stream of traffic accidents. It is critical to conduct a comprehensive analysis of this issue by utilizing voluminous data pertaining to school traffic incidents in order to safeguard the lives of faculty and students. In the case of domestic universities, fewer studies have studied knowledge graph construction methods for traffic safety incidents. In event knowledge graph construction, the reasonable release and recycling of computational resources are inefficient, and existing entity–relationship joint extraction methods are unable to deal with ternary overlapping and entity boundary ambiguity problems in relationship extraction. In response to the above problems, this paper proposes a knowledge graph construction method for university on-campus traffic safety events with improved dynamic resource scheduling algorithms and multi-layer semantic graph convolutional neural networks. The experiment’s results show that the proposed dynamic computational resource scheduling method increases GPU and CPU use by 25% and 9%. On the public dataset, the proposed data extraction model’s F1 scores for event triples increase by 1.3% on the NYT dataset and by 0.4% on the WebNLG dataset. This method can help the relevant university personnel in dealing with unexpected traffic incidents and reduce the impact on public opinion.

Conversational aspect-level quadruple sentiment analysis (DiaASQ) is proposed as a new task that aims to extract target-aspect-opinion-sentiment quadruples in dialogues. However, this task faces the problem of complex context matching and multiple utterance feature modeling, which creates difficulties in extracting quadruples from multiple intersecting utterances. To address this problem, this paper proposes a Multi-dimensional Dialogue Feature Interaction (MDFI) approach. This method models dialogue features through an interactive network structure to capture interactions between utterance features. The approach adds two layers of ResNet to achieve deep association fusion based on multi-head self-attention. It superimposes the associated features of replies, speakers, and dialogue threads layer by layer and enhances the capability of conversation representation through linear augmentation. Our model outperforms the DiaASQ benchmark model in global utterance, intra-utterance, and cross-utterance quadruple extraction. In particular, the ZH dataset shows an improvement of 7.42 in global utterance and 9.66 in cross-utterance.

Diffusion models have achieved tremendous success in modeling continuous data modalities, such as images, audio, and video, yet their application in discrete data domains (e.g., natural language) has been limited. Existing methods primarily represent discrete text in a continuous diffusion space, incurring significant computational overhead during training and resulting in slow sampling speeds. This paper introduces LaDiffuSeq, a latent diffusion-based text generation model incorporating an encoder–decoder structure. Specifically, it first employs a pretrained encoder to map sequences composed of attributes and corresponding text into a low-dimensional latent vector space. Then, without the guidance of a classifier, it performs the diffusion process for the sequence’s corresponding latent space. Finally, a pretrained decoder is used to decode the newly generated latent vectors, producing target texts that are relevant to themes and possess multiple emotional granularities. Compared to the benchmark model, DiffuSeq, this model achieves BERTScore improvements of 0.105 and 0.009 on two public real-world datasets (ChnSentiCorp and a debate dataset), respectively; perplexity falls by 3.333 and 4.562; and it effectively quadruples the text generation sampling speed.