The objective of human pose estimation (HPE) derived from deep learning aims to accurately estimate and predict the human body posture in images or videos via the utilization of deep neural networks. However, the accuracy of real-time HPE tasks is still to be improved due to factors such as partial occlusion of body parts and limited receptive field of the model. To alleviate the accuracy loss caused by these issues, this paper proposes a real-time HPE model called $${\textbf {CCAM-Person}}$$CCAM - Person based on the YOLOv8 framework. Specifically, we have improved the backbone and neck of the YOLOv8x-pose real-time HPE model to alleviate the feature loss and receptive field constraints. Secondly, we introduce the context coordinate attention module (CCAM) to augment the model’s focus on salient features, reduce background noise interference, alleviate key point regression failure caused by limb occlusion, and improve the accuracy of pose estimation. Our approach attains competitive results on multiple metrics of two open-source datasets, MS COCO 2017 and CrowdPose. Compared with the baseline model YOLOv8x-pose, CCAM-Person improves the average precision by 2.8% and 3.5% on the two datasets, respectively.

Cross-modality visible-infrared person reidentification (VI-ReID), which aims to retrieve pedestrian images captured by both visible and infrared cameras, is a challenging but essential task for smart surveillance systems. The huge barrier between visible and infrared images has led to the large cross-modality discrepancy and intraclass variations. Most existing VI-ReID methods tend to learn discriminative modality-sharable features based on either global or part-based representations, lacking effective optimization objectives. In this article, we propose a novel global-local multichannel (GLMC) network for VI-ReID, which can learn multigranularity representations based on both global and local features. The coarse- and fine-grained information can complement each other to form a more discriminative feature descriptor. Besides, we also propose a novel center loss function that aims to simultaneously improve the intraclass cross-modality similarity and enlarge the interclass discrepancy to explicitly handle the cross-modality discrepancy issue and avoid the model fluctuating problem. Experimental results on two public datasets have demonstrated the superiority of the proposed method compared with state-of-the-art approaches in terms of effectiveness.

Cross-modality visible-Infrared person re-identification (cm-ReID) is extremely challenging due to the huge modality discrepancy between RGB and IR modalities. Existing methods focus on the sample features themselves, trying to learn modality-invariant features and perform alignment to reduce the modality discrepancy in dataset-level, while the negative impact of specific features and the identity optimization are not specifically addressed. Moreover, most methods that only extracts modality-invariant appearance features cannot acquire enough discriminative matching information for identifying different persons since the information of invariant features is limited compared with original features. Accordingly, in this paper, we propose a Enhanced Invariant Feature Joint Learning Framework (EIFJLF) for cm-ReID to handle the above problems. First, we propose a specific feature confusion baseline with a novel channel-blended transformation, which confuses the visible color and infrared spectrum to alleviate the influence of specific features, so that model pays more attention to other discriminative invariant features. Second, we present an adaptive heterogeneous center loss for better identity optimization. The adaptive margin of the loss makes samples not too close to the center, avoiding losing effectiveness too early and overfitting meantime further boosting performance. Finally, we design a novel similarity feature refinement module to utilize intra-modality relations and achieve invariant information compensation. Intra-modality relations are valuable built-in invariant features and we model these relations with similarity between samples into affinities and then update the original features to achieve information compensation. EIFJLF works for more informative invariant feature learning and more stable alignment. For cm-ReID, our work is a brand new attempt. Extensive experimental results on two standard benchmarks have demonstrated superiority of the proposed method compared with state-of-the-art methods.

Traditional text-based person re-identification (ReID) techniques heavily rely on fully matched multi-modal data, which is an ideal scenario. However, due to inevitable data missing and corruption during the collection and processing of cross-modal data, the incomplete data issue is usually met in real-world applications. Therefore, we consider a more practical task termed the incomplete text-based ReID task, where person images and text descriptions are not completely matched and contain partially missing modality data. To this end, we propose a novel Prototype-guided Cross-modal Completion and Alignment (PCCA) framework to handle the aforementioned issues for incomplete text-based ReID. Specifically, we cannot directly retrieve person images based on a text query on missing modality data. Therefore, we propose the cross-modal nearest neighbor construction strategy for missing data by computing the cross-modal similarity between existing images and texts, which provides key guidance for the completion of missing modal features. Furthermore, to efficiently complete the missing modal features, we construct the relation graphs with the aforementioned cross-modal nearest neighbor sets of missing modal data and the corresponding prototypes, which can further enhance the generated missing modal features. Additionally, for tighter fine-grained alignment between images and texts, we raise a prototype-aware cross-modal alignment loss that can effectively reduce the modality heterogeneity gap for better fine-grained alignment in common space. Extensive experimental results on several benchmarks with different missing ratios amply demonstrate that our method can consistently outperform state-of-the-art text-image ReID approaches.