We focus on the problem of still image-based human action recognition, which essentially involves making prediction by analyzing human poses and their interaction with objects in the scene. Besides image-level action labels (e.g., riding, phoning), during both training and testing stages, existing works usually require additional input of human bounding boxes to facilitate the characterization of the underlying human-object interactions. We argue that this additional input requirement might severely discourage potential applications and is not very necessary. To this end, a systematic approach was developed in this paper to address this challenging problem of minimum annotation efforts, i.e., to perform recognition in the presence of only image-level action labels in the training stage. Experimental results on three benchmark data sets demonstrate that compared with the state-of-the-art methods that have privileged access to additional human bounding-box annotations, our approach achieves comparable or even superior recognition accuracy using only action annotations in training. Interestingly, as a by-product in many cases, our approach is able to segment out the precise regions of underlying human-object interactions.
8 October 2020 in IEEE Transactions on Pattern Analysis and Machine Intelligence
Accurate ground-truth pose is essential to the training of most existing head pose estimation methods. However, in many cases, the “ground truth” pose is obtained in rather subjective ways, such as asking the subjects to stare at different markers on the wall. Thus it is better to use soft labels rather than explicit hard labels to indicate the pose of a face image. This paper proposes to associate a Multivariate Label Distribution (MLD) to each image. An MLD covers a neighborhood around the original pose. Labeling the images with MLD can not only alleviate the problem of inaccurate pose labels, but also boost the training examples associated to each pose without actually increasing the total amount of training examples. Four algorithms are proposed to learn from MLD. Furthermore, an extension of MLD with the hierarchical structure is proposed to deal with fine-grained head pose estimation, which is named Hierarchical Multivariate Label Distribution (HMLD). Experimental results show that the MLD-based methods perform significantly better than the compared state-of-the-art head pose estimation algorithms. Moreover, the MLD-based methods appear much more robust against the label noise in the training set than the compared baseline methods.