Static environment is a prerequisite for most existing vision-based SLAM (simultaneous localization and mapping) systems to work properly, which greatly limits the use of SLAM in real-world environments. The quality of the global point cloud map constructed by the SLAM system in a dynamic environment is related to the camera pose estimation and the removal of noise blocks in the local point cloud maps. Most dynamic SLAM systems mainly improve the accuracy of camera localization, but rarely study on noise blocks removal. In this paper, we proposed a novel semantic SLAM system with a more accurate point cloud map in dynamic environments. We obtained the masks and bounding boxes of the dynamic objects in the images by BlitzNet. The mask of a dynamic object was extended by analyzing the depth statistical information of the mask in the bounding box. The islands generated by the residual information of dynamic objects were removed by a morphological operation after geometric segmentation. With the bounding boxes, the images can be quickly divided into environment regions and dynamic regions, so the depth-stable matching points in the environment regions are used to construct epipolar constraints to locate the static matching points in the dynamic regions. In order to verify the preference of our proposed SLAM system, we conduct the experiments on the TUM RGB-D datasets. Compared with the state-of-the-art dynamic SLAM systems, the global point cloud map constructed by our system is the best.
Face-mask occluded restoration aims at restoring the masked region of a human face, which has attracted increasing attention in the context of the COVID-19 pandemic. One major challenge of this task is the large visual variance of masks in the real world. To solve it we first construct a large-scale Face-mask Occluded Restoration (FMOR) dataset, which contains 5,500 unmasked images and 5,500 face-mask occluded images with various illuminations, and involves 1,100 subjects of different races, face orientations, and mask types. Moreover, we propose a Face-Mask Occluded Detection and Restoration (FMODR) framework, which can detect face-mask regions with large visual variations and restore them to realistic human faces. In particular, our FMODR contains a self-adaptive contextual attention module specifically designed for this task, which is able to exploit the contextual information and correlations of adjacent pixels for achieving high realism of the restored faces, which are however often neglected in existing contextual attention models. Our framework achieves state-of-the-art results of face restoration on three datasets, including CelebA, AR, and our FMOR datasets. Moreover, experimental results on AR and FMOR datasets demonstrate that our framework can significantly improve masked face recognition and verification performance.
Meng Luo, Han Zhang, Shengqiong Wu, Bobo Li, Hong Han, Hao Fei. Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). 2024.
It is essential to build good image representations for many computer vision tasks. In this study, the authors propose a hierarchical spatial pyramid max pooling method based on scale‐invariant feature transform (SIFT) features and sparse coding, which builds image representations through a hierarchical network. It includes three parts: SIFT features’ extraction, sparse coding and spatial pyramid max pooling. To mimic visual cortex, spatial pyramid max pooling is, firstly, performed on the original SIFT features in the image patches, which distils the features and extracts the most distinctive and significant feature, the SIFT‐pooled feature, in each local patch, instead of using the original SIFT features as usual. Then, a dictionary is trained using some random SIFT‐pooled features and sparse coding is performed using the trained dictionary for all SIFT‐pooled features through K‐singular value decomposition algorithm. Finally, on the sparse codes of all image patches, spatial pyramid max pooling is carried again on the image level. The image representations will be built by concatenating the pooling features of each level. The authors use the algorithm and simple linear support vector machine (SVM) for image classification on three datasets: Caltech‐101, Caltech‐256 and 15‐Scenes and the experimental results show that the authors algorithm can reach a competitive performance compared with recently published results.