Generic language models pretrained on large unspecific domains are currently the foundation of NLP. Labeled data are limited in most model training due to the cost of manual annotation, especially in domains including massive Proper Nouns such as mathematics and biology, where it affects the accuracy and robustness of model prediction. However, directly applying a generic language model on a specific domain does not work well. This paper introduces a BERT-based text classification model enhanced by unlabeled data (UL-BERT) in the LaTeX formula domain. A two-stage Pretraining model based on BERT(TP-BERT) is pretrained by unlabeled data in the LaTeX formula domain. A double-prediction pseudo-labeling (DPP) method is introduced to obtain high confidence pseudo-labels for unlabeled data by self-training. Moreover, a multi-rounds teacher–student model training approach is proposed for UL-BERT model training with few labeled data and more unlabeled data with pseudo-labels. Experiments on the classification of the LaTex formula domain show that the classification accuracies have been significantly improved by UL-BERT where the F1 score has been mostly enhanced by 2.76%, and lower resources are needed in model training. It is concluded that our method may be applicable to other specific domains with enormous unlabeled data and limited labelled data.
The Transformer-based DETR framework has achieved end-to-end set prediction, driving technological innovation in the field of object detection. However, it has two key limitations: First, the cost function overlooks semantic similarity between queries and targets, and the Hungarian algorithm fails to capture their intrinsic relationships, resulting in inaccurate matches. Second, the training phase imposes positional constraints only on positive samples and lacks an exclusion mechanism for neighboring samples, limiting detection performance. This paper analyzes the spatial proximity and semantic similarity between neighboring negative samples and positive samples, which weakens the model's ability to distinguish between positive and negative samples. To address this, a Neighboring-Aware Optimization Strategy (NAOS) is proposed and applied to the DETR framework, resulting in the NAO-DETR method. The proposed method enhances the Hungarian matching process by incorporating a semantic similarity cost between queries and targets, thereby alleviating query–target matching ambiguity. In addition, it integrates explicit supervision through geometric and coverage constraints on neighboring samples, which effectively suppresses positional interference during regression and consequently improves the accuracy of object localization. Experimental results demonstrate that NAO-DETR achieves superior performance, surpassing baseline models DINO and Deformable-DETR by 2.1% and 5.7% in mAP on COCO datasets, respectively. Moreover, NAOS demonstrates strong generalization, effectively transferring to DETR framework to improve its detection performance.
9 May 2026 in International Journal of Pattern Recognition and Artificial Intelligence
Due to the mobile energy storage character of plug-in electric vehicles (PEVs), they can be integrated with renewable energy sources (RESs) to attenuate the negative effect caused by their separate connection and increase the utility of RESs. However, unreasonable configuration of PEV infrastructures and RESs as well as inappropriate scheduling of PEVs and RESs' operation would decrease the advantage of their integration. To improve the benefit of PEVs and RESs' integration, a two-loop cooperative optimization model for configuration and scheduling of PEVs and RESs considering mobile energy storage character of PEVs is proposed. The configuration and scheduling of PEVs and RESs are optimized simultaneously by an improved particle swarm optimization (IPSO) algorithm in the model. The proposed model is validated by several case studies. Results show that by implementing the cooperative optimization, total cost of regional multi-microgrid system is significantly reduced.