Material classification is similar to texture classification and consists in predicting the material class of a surface in a color image, such as wood, metal, water, wool, or ceramic. It is very challenging because of the intra-class variability. Indeed, the visual appearance of a material is very sensitive to the acquisition conditions such as viewpoint or lighting conditions. Recent studies show that deep convolutional neural networks (CNNs) clearly outperform hand-crafted features in this context but suffer from a lack of data for training the models. In this paper, we propose two contributions to cope with this problem. First, we provide a new material dataset with a large range of acquisition conditions so that CNNs trained on these data can provide features that can adapt to the diverse appearances of the material samples encountered in real-world. Second, we leverage recent advances in multi-view learning methods to propose an original architecture designed to extract and combine features from several views of a single sample. We show that such multi-view CNNs significantly improve the performance of the classical alternatives for material classification.

Bilinear pooling is a feature aggregation step applied after the convolutional layers of a deep network and encodes a matrix of local features into a fixed-size bilinear representation. It improves performance in many image classification tasks. Since its emergence, this pooling has seen two major improvements: Compact Bilinear Pooling (CBP) and square-root normalization. Recently, the combination of these two elements has been widely studied. However, due to the lack of good normalization solutions, existing combination approaches showed less efficiency when they are plugged into different networks and less compatibility when they work with existing CBP techniques. To solve this problem, in this paper, we propose to apply Newton iterations, a fast square-root normalization method, to produce a new normalized matrix called pseudo square-rooted matrix . Subsequently, the new matrix allows a CBP technique to encode itself into a compact and normalized bilinear representation. In order to further accelerate the normalization process, our approach has two variants which can handle feature matrix extracted by different networks. Tested on three fine-grained image classification datasets, it provides competitive classification performance while consuming less computational time than other prior works.

With the rise of Convolutional Neural Network (CNN) in the recent years, image classification has shown outstanding performances in the computer vision field. Many wellknown state of the art’s CNN architectures such as the ResNet family are applying a Global Average Pooling (GAP) to reduce the number of parameters of the fully connected layers. Most of the time, this pooling operation helps to prevent overfitting but we claim that it has a serious weakness for specific images where small details are crucial to predict their category, such as material images. In this case, the details are lost in the global average, providing non accurate global features. In this paper, we propose to select the most important local features before applying the GAP. In this aim, we add a branch in the classification network that predicts the confidence the network should have in each local feature vector. The less confident features are filtered out before applying the GAP. Experimental results on three datasets show that our approach outperforms recent alternatives in terms of classification accuracy and output probability calibration.

Recognizing material from color images is still a challenging problem today. While deep neural networks provide very good results on object recognition and has been the topic of a huge amount of papers in the last decade, their adaptation to material images still requires some works to reach equivalent accuracies. Nevertheless, recent studies achieve very good results in material recognition with deep learning and we propose, in this paper, to review most of them by focusing on three aspects: material image datasets, influence of the context and ad hoc descriptors for material appearance. Every aspect is introduced by a systematic manner and results from representative works are cited. We also present our own studies in this area and point out some open challenges for future works.