Fine-grained image analysis (FGIA) is a longstanding and fundamental problem in computer vision and pattern recognition, and underpins a diverse set of real-world applications. The task of FGIA targets analyzing visual objects from subordinate categories, e.g., species of birds or models of cars. The small inter-class and large intra-class variation inherent to fine-grained image analysis makes it a challenging problem. Capitalizing on advances in deep learning, in recent years we have witnessed remarkable progress in deep learning powered FGIA. In this paper we present a systematic survey of these advances, where we attempt to re-define and broaden the field of FGIA by consolidating two fundamental fine-grained research areas – fine-grained image recognition and fine-grained image retrieval. In addition, we also review other key issues of FGIA, such as publicly available benchmark datasets and related domain-specific applications. We conclude by highlighting several research directions and open problems which need further exploration from the community.

Convolutional neural network (CNN) has demonstrated impressive ability to represent hyperspectral images and to achieve promising results in hyperspectral image classification. However, traditional CNN models can only operate convolution on regular square image regions with fixed size and weights, and thus, they cannot universally adapt to the distinct local regions with various object distributions and geometric appearances. Therefore, their classification performances are still to be improved, especially in class boundaries. To alleviate this shortcoming, we consider employing the recently proposed graph convolutional network (GCN) for hyperspectral image classification, as it can conduct the convolution on arbitrarily structured non-Euclidean data and is applicable to the irregular image regions represented by graph topological information. Different from the commonly used GCN models that work on a fixed graph, we enable the graph to be dynamically updated along with the graph convolution process so that these two steps can be benefited from each other to gradually produce the discriminative embedded features as well as a refined graph. Moreover, to comprehensively deploy the multiscale information inherited by hyperspectral images, we establish multiple input graphs with different neighborhood scales to extensively exploit the diversified spectral-spatial correlations at multiple scales. Therefore, our method is termed multiscale dynamic GCN (MDGCN). The experimental results on three typical benchmark data sets firmly demonstrate the superiority of the proposed MDGCN to other state-of-the-art methods in both qualitative and quantitative aspects.

This paper develops an unsupervised discriminant projection (UDP) technique for dimensionality reduction of high-dimensional data in small sample size cases. UDP can be seen as a linear approximation of a multimanifolds-based learning framework which takes into account both the local and nonlocal quantities. UDP characterizes the local scatter as well as the nonlocal scatter, seeking to find a projection that simultaneously maximizes the nonlocal scatter and minimizes the local scatter. This characteristic makes UDP more intuitive and more powerful than the most up-to-date method, locality preserving projection (LPP), which considers only the local scatter for clustering or classification tasks. The proposed method is applied to face and palm biometrics and is examined using the Yale, FERET, and AR face image databases and the PolyU palmprint database. The experimental results show that UDP consistently outperforms LPP and PCA and outperforms LDA when the training sample size per class is small. This demonstrates that UDP is a good choice for real-world biometrics applications

The task of traffic line detection is a fundamental yet challenging problem. Previous approaches usually conduct traffic line detection via a two-stage way, namely the line segment detection followed by a segment clustering, which is very likely to ignore the global semantic information of an entire line. To address the problem, we propose an end-to-end system called Line-CNN (L-CNN), in which the key component is a novel line proposal unit (LPU). The LPU utilizes line proposals as references to locate accurate traffic curves, which forces the system to learn the global feature representation of the entire traffic lines. We benchmark the proposed L-CNN on two public datasets including MIKKI and TuSimple, and the results suggest that L-CNN outperforms the state-of-the-art methods. In addition, L-CNN can run at approximately 30 f/s on a Titan X GPU, which indicates the practicability and effectiveness of L-CNN for real-time intelligent self-driving systems.

Vehicle trajectory prediction is a keystone for the application of the internet of vehicles (IoV). With the help of deep learning and big data, it is possible to understand the between-vehicle interaction pattern hidden in the complex traffic environment. In this paper, we propose a novel spatial-temporal dynamic attention network for vehicle trajectory prediction, which can comprehensively capture temporal and social patterns in a hierarchical manner. The social relation between vehicles is captured at each timestamp and thus retains the dynamic variation of interaction. The temporal correlation in terms of individual motion state as well as social interaction is captured by different sequential models. Furthermore, a driving intention-specific feature fusion mechanism is proposed such that the extracted temporal and social features can be integrated adaptively for the maneuver-based multi-modal trajectory prediction. Experimental results on two real-world datasets show that compared with the state-of-the-art algorithms, our proposal achieves comparable prediction performance for short-term prediction, however, works much better for long-term prediction. Additionally, various ablation analysis is provided to evaluate the effectiveness of our proposed network components. The code will be available at https://xbchen82.github.io/resource/ .

Face recognition (FR) is an active yet challenging topic in computer vision applications. As a powerful tool to represent high dimensional data, recently sparse representation based classification (SRC) has been successfully used for FR. This paper discusses the metaface learning (MFL) of face images under the framework of SRC. Although directly using the training samples as dictionary bases can achieve good FR performance, a well learned dictionary matrix can lead to higher FR rate with less dictionary atoms. An SRC oriented unsupervised MFL algorithm is proposed in this paper and the experimental results on benchmark face databases demonstrated the improvements brought by the proposed MFL algorithm over original SRC.