Scene parsing plays a crucial role when accomplishing human-robot interaction tasks. As the “eye” of the robot, RGB-D camera is one of the most important components for collecting multiview images to construct instance-oriented 3D environment semantic maps, especially in unknown indoor scenes. Although there are plenty of studies developing accurate object-level mapping systems with different types of cameras, these methods either process the instance segmentation problem in completed mapping or suffer from a critical real-time issue due to heavy computation processing required. In this paper, we propose a novel method to incrementally build instance-oriented 3D semantic maps directly from images acquired by the RGB-D camera. To ensure an efficient reconstruction of 3D objects with semantic and instance IDs, the input RGB images are operated by a real-time deep-learned object detector. To obtain accurate point cloud cluster, we adopt the Gaussian mixture model as an optimizer after processing 2D to 3D projection. Next, we present a data association strategy to update class probabilities across the frames. Finally, a map integration strategy fuses information about their 3D shapes, locations, and instance IDs in a faster way. We evaluate our system on different indoor scenes including offices, bedrooms, and living rooms from the SceneNN dataset, and the results show that our method not only builds the instance-oriented semantic map efficiently but also enhances the accuracy of the individual instance in the scene.

We propose a new method based on Functional Maps to estimate the dense point-pair relationship between single-view 3D human point clouds and template point clouds. At present, most of the relevant work is based on triangular information to estimate the correspondence, and our innovation is to process directly on the point clouds. Because the single-perspective point clouds don’t have the full human body information, these methods cannot effectively find out the correspondence. Firstly, the template is used to complete the missing human body information to obtain the full human structure, so that the Laplace-Beltrami operator (LBO) can be calculated effectively. Then, features are extracted based on the deep learning method, and geometric information was converted to spatial information. Finally, the linear function mapping is calculated to characterize the dense point-to-point correspondence.