Multimodal Relation Extraction (MRE) is a core task for constructing Multimodal Knowledge images (MKGs). Most current research is based on fine-tuning small-scale single-modal image and text pre-trained models, but we find that image-text datasets from network media suffer from data scarcity, simple text data, and abstract image information, which requires a lot of external knowledge for supplementation and reasoning. We use Multimodal Relation Data augmentation (MRDA) to address the data scarcity problem in MRE, and propose a Flexible Threshold Loss (FTL) to handle the imbalanced entity pair distribution and long-tailed classes. After obtaining prompt information from the small model as a guide model, we employ a Large Language Model (LLM) as a knowledge engine to acquire common sense and reasoning abilities. Notably, both stages of our framework are flexibly replaceable, with the first stage adapting to multimodal related classification tasks for small models, and the second stage replaceable by more powerful LLMs. Through experiments, our EMRE2llm model framework achieves state-of-the-art performance on the challenging MNRE dataset, reaching an 82.95% F1 score on the test set.

In this paper, a multi-hypothesis square-root cubature Kalman particle filter (MH-SRCKPF) is proposed for speaker tracking in noisy and reverberant environments with distributed microphone arrays. The conventional cubature Kalman particle filter (CKPF) uses the cubature Kalman filter (CKF) to generate its proposal for particle sampling. Such a proposal incorporates only one observation from a certain localization function for the state estimation, which is vulnerable to noise or reverberation, yielding the degraded tracking performance. To tackle the problem, by incorporating multiple possible observations into CKF for the proposal, a multi-hypothesis CKPF (MH-CKPF) algorithm is first developed. Furthermore, to improve the numerical stability, a multi-hypothesis square-root CKPF(MH-SRCKPF) is developed, where the state estimate and the square root of the error covariance are propagated at each time. Finally, the MH-SRCKPF is applied to the speaker tracking problems in distributed microphone arrays. Experimental results demonstrate that the proposed MH-SRCKPF outperforms the competing methods in the presence of noise and reverberation. Meanwhile, by propagating the square root of the state covariance, the proposed method exhibits attractive numerical characteristics.

The extensive use of large language models (LLMs) across various natural language processing tasks has markedly elevated the intelligence of legal systems. Despite their exceptional performance in terms of accuracy, these systems still struggle with explainability. To tackle this challenge, we propose an approach to boost the question-answering abilities of LLMs through data synthesis, focusing on Qwen-7B. By incorporating Retrieval-Augmented Generation (RAG) techniques, we enhance the system’s transparency and reliability by introducing detailed reasoning processes (CoT Prompts). Our experimental results indicate that our trained LLMs exhibit significant improvements in both answer accuracy and explainability, especially in objective evaluation tasks. Additionally, subjective assessments reveal that the model’s responses are not only precise but also highly understandable, thus boosting user confidence in the system. Overall, our research offers valuable insights and technical advancements for the development of intelligent legal question-answering systems, with significant theoretical and practical implications.

Optical braille recognition methods typically employ existing target detection models or segmentation models for the direct detection and recognition of braille characters in original braille images. However, these methods need improvement in accuracy and generalizability, especially in densely dotted braille image environments. This paper presents a two-stage braille recognition framework. The first stage is a braille dot detection algorithm based on Gaussian diffusion, targeting Gaussian heatmaps generated by the convex dots in braille images. This is applied to the detection of convex dots in double-sided braille, achieving high accuracy in determining the central coordinates of the braille convex dots. The second stage involves constructing a braille grid using traditional post-processing algorithms to recognize braille character information. Experimental results demonstrate that this framework exhibits strong robustness and effectiveness in detecting braille dots and recognizing braille characters in complex double-sided braille image datasets. The framework achieved an F1 score of 99.89% for Braille dot detection and 99.78% for Braille character recognition. Compared to the highest accuracy in existing methods, these represent improvements of 0.08% and 0.02%, respectively.

In the field of human pose estimation, heatmap-based methods have emerged as the dominant approach, and numerous studies have achieved remarkable performance based on this technique. However, the inherent drawbacks of heatmaps lead to serious performance degradation in methods based on heatmaps for smaller-scale persons. While some researchers have attempted to tackle this issue by improving the performance of small-scale persons, their efforts have been hampered by the continued reliance on heatmap-based methods. To address this issue, this paper proposes the SSA Net, which aims to enhance the detection accuracy of small-scale persons as much as possible while maintaining a balanced perception of persons at other scales. SSA Net utilizes HRNetW48 as a feature extractor and leverages the TDAA module to enhance small-scale perception. Furthermore, it abandons heatmap-based methods and instead adopts coordinate vector regression to represent keypoints. Notably, SSA Net achieved an AP of 77.4% on the COCO Validation dataset, which is superior to other heatmap-based methods. Additionally, it achieved highly competitive results on the Tiny Validation and MPII datasets as well.

Self-attention mechanisms are widely used in current encoder-decoder frameworks of image-captioning tasks. These mechanisms use the transformer module as the basic unit to compute the similarity between query and key vectors, and normalize them to obtain the attention distribution matrix, weighting and reconstructing the rich semantic query vectors. However, a single attention distribution matrix cannot express more complex intrinsic relations between query and key vectors. Some very important intrinsic relations can thus become lost. We therefore propose a hybrid attention distribution (HAD) that allows multiple distributions to be reconstructed to express deeper internal relations and avoids a single shallow attention distribution. In addition, to reduce the number of model parameters, we factorize the word-embedding matrix. This effectively increases the training efficiency and prevents the metrics from decreasing. We extensively evaluate our proposals using the MS-COCO image-captioning dataset. Our results outperform existing state-of-the-art methods on some metrics.