This paper focuses on fine-grained object classification using recognized scene text in natural images. While the state-of-the-art relies on visual cues only, this paper is the first work which proposes to combine textual and visual cues. Another novelty is the textual cue extraction. Unlike the state-of-the-art text detection methods, we focus more on the background instead of text regions. Once text regions are detected, they are further processed by two methods to perform text recognition, i.e., ABBYY commercial OCR engine and a state-of-the-art character recognition algorithm. Then, to perform textual cue encoding, bi- and trigrams are formed between the recognized characters by considering the proposed spatial pairwise constraints. Finally, extracted visual and textual cues are combined for fine-grained classification. The proposed method is validated on four publicly available data sets: ICDAR03, ICDAR13, Con-Text, and Flickr-logo. We improve the state-of-the-art end-to-end character recognition by a large margin of 15% on ICDAR03. We show that textual cues are useful in addition to visual cues for fine-grained classification. We show that textual cues are also useful for logo retrieval. Adding textual cues outperforms visual- and textual-only in fine-grained classification (70.7% to 60.3%) and logo retrieval (57.4% to 54.8%).

We present a novel architecture for manipulating facial expressions, head poses, and lighting conditions from a single monocular image. Recent methods based on Generative Adversarial Networks show promising results in expression manipulation. However, the variation is either defined by a limited number of classes or not well suitable for explicit manipulation of different attributes such as pose and lighting conditions. Besides, state-of-the-art methods are mostly focused on frontal faces. Therefore, in this paper, a new Generative Adversarial Network architecture is proposed by explicitly conditioning on the appearance image space which is the product of direct manipulation of facial expressions, light and pose conditions of the face model in 3D space. In addition, the method provides face annotations (i.e. facial units, landmarks, etc.) and only requires video sequences for training. Therefore, it is unsupervised. Unlike other face manipulation methods, the proposed method does not require target specific training. Large scale experiments show that our method outperforms state-of-the-art methods for different scenarios.

Latest advances of deep learning paradigm and 3D imaging systems have raised the necessity for more complete datasets that allow exploitation of facial features such as pose, gender or age. In our work, we propose a new facial dataset collected with an innovative RGB–D multi-camera setup whose optimization is presented and validated. 3DWF includes 3D raw and registered data collection for 92 persons from low-cost RGB–D sensing devices to commercial scanners with great accuracy. 3DWF provides a complete dataset with relevant and accurate visual information for different tasks related to facial properties such as face tracking or 3D face reconstruction by means of annotated density normalized 2K clouds and RGB–D streams. In addition, we validate the reliability of our proposal by an original data augmentation method from a massive set of face meshes for facial landmark detection in 2D domain, and by head pose classification through common Machine Learning techniques directed towards proving alignment of collected data.