Background subtraction is an essential step in many computer vision tasks. In this article, we put forth a unique attempt to detect the local changes in challenging video scenes by exploring the capabilities of an encoder-decoder type network that employs a modified ResNet-152 architecture with a multi-scale features extraction framework. The proposed encoder network consists of a modified ResNet-152 network where the initial two blocks are freeze and the weights of the last blocks are learned using a transfer learning mechanism. The said encoder can reduce the computational complexity of the proposed model and extract fine as well as coarse-scale features. We have proposed a multiscale features extraction (MFE) mechanism block which is a hybridization of pyramidal pooling architecture (PPA), and various atrous convolutional layers where the high-level features from the encoder network are utilized to extract features at various scales. The use of PPA in the MFE block preserves maximum value in every pooling area, to retain the contextual relationship between the pixels in the complex video frames that can handle various challenging scenes. The proposed decoder network consists of stacked transposed convolution layers that learn a mapping from feature space to image space, predicting a score map. Then, a threshold is applied on the score map to get the binary class labels as the background and foreground. The shortcut connections followed by global average pooling (GAP) drive the low-level feature coefficients from the encoder network to the decoder network to enhance the feature representation. The performance of the proposed scheme is validated by testing it against thirty-one state-of-the-art techniques. The results obtained by the proposed method are corroborated qualitatively as well as quantitatively. Further, the efficacy of the proposed algorithm is verified with an unseen video setup and is found to provide better performance.

Quality assessment of 3D-synthesized images has traditionally been based on detecting specific categories of distortions such as stretching, black-holes, blurring, etc. However, such approaches have limitations in accurately detecting distortions entirely in 3D synthesized images affecting their performance. This work proposes an algorithm to efficiently detect the distortions and subsequently evaluate the perceptual quality of 3D synthesized images. The process of generation of 3D synthesized images produces a few pixel shift between reference and 3D synthesized image, and hence they are not properly aligned with each other. To address this, we propose using morphological operation (opening) in the residual image to reduce perceptually unimportant information between the reference and the distorted 3D synthesized image. The residual image suppresses the perceptually unimportant information and highlights the geometric distortions which significantly affect the overall quality of 3D synthesized images. We utilized the information present in the residual image to quantify the perceptual quality measure and named this algorithm as Perceptually Unimportant Information Reduction (PU-IR) algorithm. At the same time, the residual image cannot capture the minor structural and geometric distortions due to the usage of erosion operation. To address this, we extract the perceptually important deep features from the pre-trained VGG-16 architectures on the Laplacian pyramid. The distortions in 3D synthesized images are present in patches, and the human visual system perceives even the small levels of these distortions. With this view, to compare these deep features between reference and distorted image, we propose using cosine similarity and named this algorithm as Deep Features extraction and comparison using Cosine Similarity (DF-CS) algorithm. The cosine similarity is based upon their similarity rather than computing the magnitude of the difference of deep features. Finally, the pooling is done to obtain the objective quality scores using simple multiplication to both PU-IR and DF-CS algorithms. Our source code is available online: https://github.com/sadbhawnathakur/3D-Image-Quality-Assessment .

Several research works have been developed in the area of action recognition. Unfortunately, when these algorithms are applied to low-light or dark videos, their performances are highly affected and found to be very poor or fall rapidly. To address the issue of improving the performance of action recognition in dark or low-light videos; in this article, we have developed an efficient deep 3D CNN based action recognition model. The proposed algorithm follows two-stages for action recognition. In the first stage, the low-light videos are enhanced using Zero-Reference Deep Curve Estimation (Zero-DCE), followed by the min-max sampling algorithm. In the latter stage, we propose an action classification network to recognize the actions in the enhanced videos. In the proposed action classification network, we explored the capabilities of the $R(2+1)D$ for spatio-temporal feature extraction. The model's overall generalization performance depends on how well it can capture long-range temporal structure in videos, which is essential for action recognition. So we have used a Graph convolutional network (GCN) on the top of R(2+1)D as our video feature encoder which captures long-term temporal dependencies of the extracted features. Finally, a Bidirectional Encoder Representations from Transformers (BERT) is adhered to classify the actions from the 3D features extracted from the enhanced video scenes. The effectiveness of the proposed action recognition scheme is verified on ARID V1.0 and ARID V1.5 datasets. It is observed that the proposed algorithm is able to achieve $96.60\%$ and $99.88\%$ as Top-1 and Top-5 accuracy, respectively, on ARID V1.0 dataset. Similarly, on ARID V1.5, the proposed algorithm is able to achieve $86.93\%$ and $99.35\%$ as Top-1 and Top-5 accuracies, respectively. To corroborate our findings, we have compared the results obtained by the proposed scheme with those of fifteen state-of-the-art action recognition techniques.

Haze is an intrusion element that disrupts color fidelity and contrast of outdoor natural images, affecting their perceptual quality. The differential characteristics of hazy images compared to other natural images restrict the generalization of existing image quality assessment (IQA) algorithms. At the same time, efficient IQA algorithms for predicting the perceptual quality of naturally hazed images have not been proposed in the literature due to lack of a relevant dataset. To address this, we build the IIT-JMU Hazy Image Dataset comprising of 1000 high-definition hazy natural images consisting of diverse categories such as landscape, forests, roads, seascapes, and cityscapes, along with their subjective quality ratings. We present an analysis of existing natural-scene-statistics-based IQA algorithms on hazy natural images. In this article, we propose a convolutional-neural-network-based quality assessment algorithm for hazy natural images along with an IQA metric called deep learning-based haze perceptual quality evaluator (DLHPQE). The proposed DLHPQE efficiently predicts the perceptual quality of hazy natural images without a reference. Our results demonstrate that the DLHPQE outperforms existing state-of-the-art no-reference IQAs in terms of several performance parameters such as Pearson linear correlation coefficient, Spearman rank-order correlation coefficient, Kendall's rank-order correlation coefficient, and root-mean-square error.

Existing Quality Assessment (QA) algorithms consider identifying “black-holes” to assess perceptual quality of 3D-synthesized views. However, advancements in rendering and inpainting techniques have made black-hole artifacts near obsolete. Further, 3D-synthesized views frequently suffer from stretching artifacts due to occlusion that in turn affect perceptual quality. Existing QA algorithms are found to be inefficient in identifying these artifacts, as has been seen by their performance on the IETR dataset. We found, empirically, that there is a relationship between the number of blocks with stretching artifacts in view and the overall perceptual quality. Building on this observation, we propose a Convolutional Neural Network (CNN) based algorithm that identifies the blocks with stretching artifacts and incorporates the number of blocks with the stretching artifacts to predict the quality of 3D-synthesized views. To address the challenge with existing 3D-synthesized views dataset, which has few samples, we collect images from other related datasets to increase the sample size and increase generalization while training our proposed CNN-based algorithm. The proposed algorithm identifies blocks with stretching distortions and subsequently fuses them to predict perceptual quality without reference, achieving improvement in performance compared to existing no-reference QA algorithms that are not trained on the IETR dataset. The proposed algorithm can also identify the blocks with stretching artifacts efficiently, which can further be used in downstream applications to improve the quality of 3D views. Our source code is available online: https://github.com/sadbhawnathakur/3D-Image-Quality-Assessment .