Visual affordance recognition is an important research topic in robotics, human-computer interaction, and other computer vision tasks. In recent years, deep learning-based affordance recognition methods have achieved remarkable performance. However, there is no unified and intensive survey of these methods up to now. Therefore, this paper reviews and investigates existing deep learning-based affordance recognition methods from a comprehensive perspective, hoping to pursue greater acceleration in this research domain. Specifically, this paper first classifies affordance recognition into five tasks, delves into the methodologies of each task, and explores their rationales and essential relations. Second, several representative affordance recognition datasets are investigated carefully. Third, based on these datasets, this paper provides a comprehensive performance comparison and analysis of the current affordance recognition methods, reporting the results of different methods on the same datasets and the results of each method on different datasets. Finally, this paper summarizes the progress of affordance recognition, outlines the existing difficulties and provides corresponding solutions, and discusses its future application trends.
Although deep networks based methods outperform traditional 3D reconstruction methods which require multiocular images or class labels to recover the full 3D geometry, they may produce incomplete recovery and unfaithful reconstruction when facing occluded parts of 3D objects. To address these issues, we propose Depth-preserving Latent Generative Adversarial Network (DLGAN) which consists of 3D Encoder-Decoder based GAN (EDGAN, serving as a generator and a discriminator) and Extreme Learning Machine (ELM, serving as a classifier) for 3D reconstruction from a monocular depth image of an object. Firstly, EDGAN decodes a latent vector from the 2.5D voxel grid representation of an input image, and generates the initial 3D occupancy grid under common GAN losses, a latent vector loss and a depth loss. For the latent vector loss, we design 3D deep AutoEncoder (AE) to learn a target latent vector from ground truth 3D voxel grid and utilize the vector to penalize the latent vector encoded from the input 2.5D data. For the depth loss, we utilize the input 2.5D data to penalize the initial 3D voxel grid from 2.5D views. Afterwards, ELM transforms float values of the initial 3D voxel grid to binary values under a binary reconstruction loss. Experimental results show that DLGAN not only outperforms several state-of-the-art methods by a large margin on both a synthetic dataset and a real-world dataset, but also predicts more occluded parts of 3D objects accurately without class labels.
13 November 2019 in IEEE Transactions on Multimedia
Pose tracking from range image sequences remains a difficult task due to strong noise and serious self-occlusion of human body. Existing work either rely on extremely large and precisely annotated datasets, or rely on accurate human mesh model and GPU acceleration. In this paper, we propose an unsupervised real-time framework of pose tracking from range image sequences. Our framework consists of a visible hybrid model (VHM), a componentwise correspondence optimization (CCO) and a dynamic database lookup (DDL). VHM consists of component sphere sets and component visible spherical point sets which exhibits both simplicity and high accuracy. CCO converts the matching between VHM and input point cloud into several subproblems regarding local rotations of components and a global translation of body abdominal joint, each of which has an efficient closed form solution. DDL is designed to recover correct pose when tracking fails, which effectively mitigates accumulative error during tracking. Experiments on SMMC, PDT, EVAL datasets indicate that our framework not only achieves better or competitive precision compared with state-of-the-art methods, but also produces real-time efficiency in personal computers without GPU acceleration.
28 May 2021 in Frontiers of Information Technology & Electronic Engineering
Three-dimensional (3D) reconstruction of shapes is an important research topic in the fields of computer vision, computer graphics, pattern recognition, an