23 October 2020 in IEEE Transactions on Neural Networks and Learning Systems
Despite the success of batch normalization (BatchNorm) and a plethora of its variants, the exact reasons for its success are still shady. The original BatchNorm article explained it as a mechanism that reduces the internal covariate shift (ICS), i.e., the distribution shifts in the input of the layers during training. Recently, some articles manifested skepticism on this hypothesis and provided alternative explanations for the success of BatchNorm, such as the applicability of very high learning rates and the ability to smooth the landscape in optimization. In this work, we counter these alternative arguments by demonstrating the importance of reduction in ICS following an empirical approach. We demonstrated various ways to achieve the abovementioned alternative properties without any performance boost. In this light, we explored the importance of different BatchNorm parameters (i.e., batch statistics and affine transformation parameters) by visualizing their effectiveness in the performance and analyzed their connections with ICS. Afterward, we showed a different normalization scheme that fulfills all the alternative explanations except reduction in ICS. Despite having all the alternative properties, we observed its poor performance, which nullifies the alternative claims, rather signifies the importance of the ICS reduction. We performed comprehensive experiments on many variants of BatchNorm, finding that all of them similarly reduce ICS.
26 April 2018 in IEEE Transactions on Affective Computing
Currently available local feature descriptors used in facial expression recognition at times suffer from unstable feature descriptions, especially in the presence of weak and distorted edges due to noise, limiting their performances. We propose a novel local descriptor named Neighborhood-aware Edge Directional Pattern (NEDP) to overcome such limitations. Instead of relying solely on the local neighborhood to describe the feature around a pixel, as done by the existing local descriptors, NEDP examines the gradients at the target (center) pixel as well as its neighboring pixels to explore a wider neighborhood for the consistency of the feature in spite of the presence of subtle distortion and noise in local region. We introduce template-orientations for the neighboring pixels, which give importance to the gradients in consistent edge directions, prioritizing the specific neighbors falling in the direction of the local edge to represent the shape of the local textures, unambiguously. Moreover, due to the effective management of the featureless regions, no such region is erroneously encoded as a feature by NEDP. Experiments of the performances for person-independent recognition on benchmark expression datasets also show that NEDP performs better than other existing descriptors, and thereby, improves the overall performance of facial expression recognition.
18 April 2017 in IEEE Transactions on Information Forensics and Security
An appropriate aging description from face image is the prime influential factor in human age recognition, but still there is an absence of a specially engineered aging descriptor, which can characterize discernible facial aging cues (e.g., craniofacial growth, skin aging) from a detailed and more finer point of view. To address this issue, we propose a local face descriptor, directional age-primitive pattern (DAPP), which inherits discernible aging cue information and is functionally more robust and discriminative than existing local descriptors. We introduce three attributes for coding the DAPP description. First, we introduce Age-Primitives encoding aging related to the most crucial texture primitives, yielding a reasonable and clear aging definition. Second, we introduce an encoding concept dubbed as Latent Secondary Direction, which preserves compact structural information in the code avoiding uncertain codes. Third, a globally adaptive thresholding mechanism is initiated to facilitate more discrimination in a flat and textured region. We apply DAPP on separate age group recognition and age estimation tasks. Applying the same approach to both of these tasks is seldom explored in the literature. Carefully conducted experiments show that the proposed DAPP description outperforms the existing approaches by an acceptable margin.
22 January 2019 in Turkish Journal of Electrical Engineering and Computer Sciences
17 August 2020 in IEEE Access
Deep Learning techniques have been successfully used to solve a wide range of computer vision problems. Due to their high computation complexity, specialized hardware accelerators are being proposed to achieve high performance and efficiency for deep learning-based algorithms. However, soft errors, i.e., bit flipping errors in the layer output, are often caused due to process variation and high energy particles in these hardware systems. These can significantly reduce model accuracy. To remedy this problem, we propose new algorithms that effectively reduce the impact of errors, thus keeping high accuracy. We firstly propose to incorporate an Error Correction Layer (ECL) into neural networks where convolution is performed multiple times in each layer and majority reporting is conducted for the outputs at bit level. We found that ECL can eliminate most errors while bypassing the bit-error when the bits at the same position are corrupted multiple times under the simulated condition. In order to solve this problem, we analyze the impact of errors depending on the position of bits, thus observing that errors in most significant bit (MSB) positions tend to severely corrupt the output of the network compared to the errors in the least significant bit (LSB) positions. According to this observation, we propose a new specialized activation function, called Piece-wise Rectified Linear Unit (PwReLU), which selectively suppresses errors depending on the bit positions, resulting in an increased model resistance against the errors. Compared to existing activation functions, the proposed PwReLU outperforms with large accuracy margins of up-to 20% even with very high bit error rates (BERs). Our extensive experiments show that the proposed ECL and PwReLU work in a complementary manner, achieving comparable accuracy to the error-free networks even at a severe BER of 0.1% on CIFAR10, CIFAR100, and ImageNet.
18 May 2020 in IEEE Transactions on Affective Computing
Facial expression recognition has been studied broadly, and several works using local micro-pattern descriptors have obtained significant results. There are, however, open questions: how to design a discriminative and robust feature descriptor?, how to select expression-related most influential features?, and how to represent the face descriptor exploiting the most salient parts of the face? In this paper, we address these three issues to achieve better performance in recognizing facial expressions. First, we propose a new feature descriptor, namely Local Shape Pattern (LSP), that describes the local shape structure of a pixel's neighborhood based on the prominent directional information by analyzing the statistics of the neighborhood gradient, which allows it to be robust against subtle local noise and distortion. Furthermore, we propose a selection strategy for learning the influential codes being active in the expression affiliated changes by selecting them exhibiting statistical dominance and high spatial variance. Lastly, we learn the size of the salient facial blocks to represent the facial description with the notion that changes in expressions vary in size and location. We conduct person-independent experiments in existing datasets after combining above three proposals, and obtain an improved performance for the facial expression recognition task.