Falls are the main cause of accidental injuries, and even death among elderly people, especially those who live alone in their homes. The absence of a reliable fall detection system has long been a serious problem for home health monitoring. A video surveillance system can be used to monitor elderly people at home to detect falls, but the traditional implementation of such intelligent detection falls short of personal privacy-related considerations; additionally, many people do not want to be watched in their homes. To solve this problem, we propose a fall detection system with visual shielding that can ensure the safety of elderly people in their homes while preserving their personal privacy. Multilayer compressed sensing is first used to achieve visually shielded video frames. By combining low-rank sparse decomposition theory with the improved local binary pattern on the three orthogonal planes, the object features are extracted from the shielded video frames. Finally, to compensate for the information lost in the compressed video to a certain extent, a private information-embedded classification model is proposed to identify fall-related behavior. The experimental results on two public fall datasets show that the proposed method delivers impressive accuracy and a low error rate while effectively distinguishing between fall- and nonfall-related behaviors in videos.

Text-to-image person re-identification (ReID) is a common subproblem in the field of person re-identification and image-text retrieval. Recent approaches generally follow the structure of a dual-stream network, extracting image and text features. There is no deep interaction between images and text in this approach, making it difficult for the network to learn a highly semantic feature representation. In addition, for both image data and text data, the feature extraction process is modeled in a regular way, such as using Transformer to extract sequence embeddings. However, this type of modeling disregards the inherent relationships among multimodal input embeddings. A more flexible approach to mining multimodal data, which uniformly treats the data as graphs, is proposed. In this way, the extraction and interaction of multimodal information are accomplished by means of messages passing between graph nodes. First, a unified multimodal feature extraction and fusion network is proposed based on the graph convolutional network, which enables the progression of multimodal information from ‘local’ to ‘global’. Second, an asymmetric multilevel alignment module, which focuses on more accurate ‘local’ information from a ‘global’ perspective, is proposed to progressively divide the multimodal information at each level. Last, a cross-modal representation matching strategy based on similarity distribution and mutual information is proposed to achieve cross-modal alignment. The proposed algorithm in this paper is simple and efficient, and the testing results on three public datasets (CUHK-PEDES, ICFG-PEDES and RSTPReID) show that it can achieve SOTA-level performance.

The austere challenge of visual object tracking is to find the target to be tracked in various noise interference and obtain its accurate bounding box coordinates. Recently, the object tracking technology based on the Siamese network has made great breakthroughs, and more and more Siamese network trackers have been proposed with superior performance. They still have some shortcomings. To this end, a new Multi-Stage visual tracking algorithm with Siamese Anchor-Free Proposal Network (MS-SiamAFPN) is proposed in this paper. The algorithm is a three-stage Siamese network tracker composed of Feature Extraction and Fusion (FEF) sub-network, Classification and Regression (CR) sub-network, Validation and Regression (VR) sub-network in series. Firstly, the Anchor-Free Proposal Network (AFPN) module is designed in the CR stage, which can make full use of positive and negative samples for training while reducing neural network parameters. Secondly, aim to achieve better robustness and recognizability in the VR stage, on the one hand, a novel Feature Purification (FP) module is designed, which can automatically select the important channels, and extract the features of irregular regions on the input fusion features, so as to strengthen the representation ability of image features. On the other hand, the target recognition and position regression are regarded as different processing tasks, and the recognition score and position fine-tuning of candidate targets are obtained by newly designing the Dual-Branch Network (DBN) structure, thereby avoiding feature ambiguity. Due to the synergy of the above these innovations, MS-SiamAFPN has obtained a large performance improvement, and achieved SOTA performance in multiple public dataset benchmarks.

Facial expression recognition (FER) in the wild unavoidably sufferes from the effects of face posture, illumination and partial occlusion. In this paper, we attempt to alleviate the above negative effects and improve the performance of FER in the wild based on 3D face feature reconstruction and learning. 3D face reconstruction not only can effectively make up for the facial apparent information missing inform a 2D face images but can also extract accurate 3D facial geometric information in self-occlusion and extreme illumination scenarios. Therefore, we propose a novel end-to-end trainable 3D face feature reconstruction and learning network(3DF-RLN) is proposed for FER in the wild. In 3DF-RLN, the 2D implicitly frontalized face apparent data and 3D facial landmarks are reconstructed by a 3D face reconstruction module and input to two feature extraction pathways. The appearance pathway learns apparent features from the reconstructed 2D face apparent data using a convolutional neural network. The geometry pathway learns the geometric features from the reconstructed 3D facial landmarks using a graph convolutional network. Finally, FER is achieved via the fusion of the two pathways. Extensive experiments were conducted to evaluate the proposed method with three benchmark databases including Multi-PIE, RAF-DB and AffectNet. The results show that the proposed 3DF-RLN model has better FER performance, both in the lab and in the wild. In addition, the face graph from the geometry pathway reveals the correlations between facial landmarks in FER.

In order to solve the problems of unbalanced sample data and the lack of consideration of temporal information in existing Siamese-based trackers, this paper proposes a Siamese recurrent neural network and region proposal network (Siamese R-RPN), which can be trained in an end-to-end manner. Siamese R-RPN is consisted of Siamese network, recurrent neural network and region proposal network. Image features extracted by the Siamese network are strengthened by the channel and spatial attention mechanisms, and are sent to the RPN for classification and regression. Temporal information is processed by a recurrent neural network-based Long Short-Term Memory (LSTM) to predict the rough location of the target, it is mapped to the anchor feature map of the RPN for anchor selection. This makes the positive and negative samples participating in the training procedure to become more balanced and representative. Because of the collaborative use of temporal and spatial information, the tracker proposed in this paper has achieved state-of-the-art performance on three large tracking benchmarks—OTB 2015, VOT2016 and VOT 2018—where this verifies its effectiveness.

Low-light image enhancement is a crucial area of research in computer vision, aimed at recovering normally exposed images from low-light images to facilitate high-level vision tasks such as target detection, tracking, and recognition. However, the convolutional neural networks commonly used in this field have a bias towards extracting low-frequency local structural features in the spatial domain, resulting in unclear texture details in the enhanced images. To address this limitation, this paper proposes a novel frequency-domain perception-based model, called DSFPNet. This model has three unique features. First, the recursive The Hadamard product method is used to model long-range relationships of the Transformer in the feature extraction. Second, a bilateral gating mechanism is added to the feedforward network to filter out useless information and improve the nonlinear modeling capability of the module. Third, cross-layer connectivity of low-frequency features is added to maintain the structural stability of features at different levels. In addition, the proposed model uses an EFF (Enhancing Frequency Features) module to extract frequency features and selectively fuse the three high frequencies. The fused features are then aligned and reconstructed with the low-frequency features to obtain the final frequency features, which helps the model recover enhanced image texture details. Through a large number of experimental results on the two public datasets, the proposed DSFPNet model shows excellent performances and outperforms many existing state-of-the-art methods. The model exhibits good potential and is expected to advance the field of low-light image enhancement in computer vision.