Real-time 3D reconstruction is one of the current popular research directions of computer vision, and it has become the core technology in the fields of virtual reality, industrialized automatic systems, and mobile robot path planning. Currently, there are three main problems in the real-time 3D reconstruction field. Firstly, it is expensive. It requires more varied sensors, so it is less convenient. Secondly, the reconstruction speed is slow, and the 3D model cannot be established accurately in real time. Thirdly, the reconstruction error is large, which cannot meet the requirements of scenes with accuracy. For this reason, we propose a real-time 3D reconstruction method based on monocular vision in this paper. Firstly, a single RGB-D camera is used to collect visual information in real time, and the YOLACT++ network is used to identify and segment the visual information to extract part of the important visual information. Secondly, we combine the three stages of depth recovery, depth optimization, and deep fusion to propose a three-dimensional position estimation method based on deep learning for joint coding of visual information. It can reduce the depth error caused by the depth measurement process, and the accurate 3D point values of the segmented image can be obtained directly. Finally, we propose a method based on the limited outlier adjustment of the cluster center distance to optimize the three-dimensional point values obtained above. It improves the real-time reconstruction accuracy and obtains the three-dimensional model of the object in real time. Experimental results show that this method only needs a single RGB-D camera, which is not only low cost and convenient to use, but also significantly improves the speed and accuracy of 3D reconstruction.
Most current pose estimation methods have a high resource cost that makes them unusable in some resource-limited devices. To address this problem, we propose an ultra-lightweight end-to-end pose distillation network, which applies some helpful techniques to suitably balance the number of parameters and predictive accuracy. First, we designed a lightweight one-stage pose estimation network, which learns from an increasingly refined sequential expert network in an online knowledge distillation manner. Then, we constructed an ultra-lightweight re-parameterized pose estimation subnetwork that uses a multi-module design with weight sharing to improve the multi-scale image feature acquisition capability of the single-module design. When training was complete, we used the first re-parameterized module as the deployment network to retain the simple architecture. Finally, extensive experimental results demonstrated the detection precision and low parameters of our method.
Accurate lightweight pose estimation is still a difficult challenging task influenced by different human poses and various complex backgrounds in 2D human images. To address the above problems, we propose a Lightweight Single-branch Pose Distillation network, termed LSPD, which is a lightweight powerful fully convolutional pose network that can be executed quickly with a low computational cost for accurate pose estimation. First, we introduced an efficient end-to-end pose distillation sequence framework, which utilizes a small number of lightweight and strong pose estimation stages to effectively transfer the pose knowledge of our teacher model. Second, we constructed a compact and strong pose estimation stage that uses a type of lightweight multi-scale residual block to enhance the image features and the image-dependent spatial features representation ability of the model. At the same time, it reduces the computational cost. Finally, when training is complete, we used the backbone network and the first student stage as the simple architecture to deploy. Extensive experiments demonstrated that the proposed method obtains excellent performance with high accuracy and low model parameters.
Semantic matching research is the cornerstone of research in the fields of natural language similarity measurement and sensor ontology matching (OM). In the existing Chinese semantic matching methods, there are some shortcomings, such as the single dimension of semantic expression, the insufficient expression of context semantic relations, and the insufficient interaction of semantic information between different sentences. This paper proposes a Chinese semantic matching algorithm based on RoBERTa-wwm-ext with Siamese interaction and fine-tuning representation (RSIFR). The RSIFR model initializes the model with RoBERTa-wwm-ext as a vector of text. Firstly, a Siamese structure with embedded soft alignment attention mechanism and BiLSTM is constructed to realize the information interaction between two sentences. Secondly, LSTM-BiLSTM network structure is constructed to enhance the expression of semantic logic before and after sentences. Then, build a training model with fine-tuning mechanism. Fine tune the text’s eigenvector parameters through label supervision. Finally, the fusion vectors of the sentence pairs are inserted into the MLP network layer, resulting in semantic matching results. RSIFR model starts from a variety of dimensions, strengthens the expression ability of vectors to text semantic relations, deeply mines the semantic similarities and differences between different sentences, and generally improves the Chinese semantic matching performance. Experiments on the public dataset LCQMC show that our model outperforms existing Chinese semantic matching models.
In recent years, object detection has been widely used in various fields such as face detection, remote sensing image detection and pedestrian detection. Due to the complex environment in the actual scene, we need to fully obtain the feature information in the image to improve the accuracy of object detection. This paper proposes an object detection algorithm based on coordinate attention and contextual feature enhancement. We design a multi-scale attention feature pyramid network, which first uses multi-branch atrous convolution to capture multi-scale context information, and then fuses the coordinate attention mechanism to embed location information into channel attention, and finally uses a bidirectional feature pyramid structure to effectively fuse high-level features and low-level features. We also adopt the GIoU loss function to further improve the accuracy of object detection. The experimental results show that the proposed method has certain advantages compared with other detection algorithms in the PASCAL VOC datasets.
1 October 2025 in IEICE Transactions on Information and Systems
Image-text retrieval (ITR) aims at querying one type of data based on a given another type of data. The main challenge is mapping images and texts to a common space. Although existing methods obtain excellent performance on ITR tasks, they also have the drawbacks of weak information interaction and insufficient capture of deeper associative relationships. To address these problems, we propose CDISA: a Cross-modal Deep Interaction and Semantic Aligning method by combining vision-language pre-training model with semantic feature extraction capabilities. Specifically, we first design a cross-modal deep interaction module to enhance the interaction of image and text features by performing deep interaction matching computations. Secondly, to align the image and text features, bidirectional cosine matching is proposed to improve the differentiation of bimodal data within the feature space. We propose arguably the extensive experimental evaluation against recent state-of-the-art ITR methods on three datasets which include Wikipedia, Pascal-Sentence and NUS-WIDE.