16 October 2017 in IEEE Transactions on Multimedia
According to the types of the used features, state-ofthe-art methods for blind image quality assessment (BIQA) typically have two categories. The first category utilizes handcrafted natural scene statistics (NSS) features which are derived based on the statistical regularity of natural images. The second category utilizes codebook-based features which are obtained by feature encoding over a pre-constructed codebook. However, several problems need to be addressed in the existing codebook-based BIQA methods. First, dimension of the involved codebook-based features is too high. Second, codebooks are usually constructed by unsupervised learning algorithms. To address these problems, we propose a novel codebook-based BIQA method by optimizing multi-stage discriminative dictionaries (MSDDs). To be more specific, MSDDs are learned by performing the label consistent K-SVD (LC-KSVD) algorithm in a stage-by-stage manner. In each stage, a new quality consistency constraint called “qualitydiscriminative regularization” term is introduced and incorporated into the traditional reconstruction error term to form a unified objective function which can be effectively solved by LC-KSVD. Then, the latter stage takes the reconstruction residual data in the former stage as input based on which LC-KSVD is repeatedly performed until the final stage is reached. Once the MSDDs are learned, multi-stage feature encoding (MSFE) is performed to extract feature codes. Finally, the feature codes are concatenated across all stages and spatially aggregated over the entire image for quality prediction via a typical regression module. The proposed method has been extensively evaluated on five IQA databases and the experimental results has well confirmed its superiority over the existing relevant BIQA methods.
23 January 2007 in Wireless Communications and Mobile Computing
15 March 2010 in IEEE Transactions on Circuits and Systems for Video Technology
A motion-compensated deinterlacing scheme based on hierarchical motion analysis is presented. According to deinterlacing steps, our contribution can be divided into four parts: motion estimation, motion state analysis, motion consistency analysis, and finer-grained interpolation. In motion estimation, we introduce a Gaussian noise model for choosing the best motion vector for each block, and make a tradeoff between utilizing previous de-interlaced frames and avoiding error propagation. A directional interpolation method is also introduced in this part for backward fields. In motion state analysis, we define two motion states for each pixel, thus achieve a compromise between traditional block-based strategies and the extreme pixel-based case. In motion consistency analysis, we propose to measure both the motion vector consistency and the motion state consistency in order to determine whether the previous two parts should be performed again with a different block size. In finer-grained interpolation, we utilize a combination of recursive median filters to generate the final results. Experimental results show that all of the proposed techniques are effective, either objectively or subjectively. As a result, we can achieve much higher image quality, with an average gain of about 1.83 dB in terms of peak signal-to-noise ratio. Moreover, the increased computation complexity is marginal.
20 November 2020 in IEEE Transactions on Multimedia
Screen content images (SCIs) have been rapidly and widely applied in interactive multimedia applications. The problem of quality assessment for SCIs is an interesting research topic. Most of the existing methods use subjective and independent features in gray domain to predict the image quality, which can't comprehensively characterize the image properties or lack unified mathematical explanation for SCIs. To address these problems, we propose a novel blind quality assessment method based on macro-micro modeling of tensor domain dictionary for SCIs in this paper. In the proposed method, the tensor decomposition is explored first to avoid the loss of color information, and then a target dictionary is learned more effectively with the principal components. Furthermore, a macro-micro model is established to characterize the micro and macro features in the target dictionary space, which is capable of providing a systematic mathematical interpretation for feature extraction. For the micro features, a log-normal pooling scheme is designed to enhance the effectiveness of feature aggregation by analyzing the particularity of the statistical distribution of sparse codes. Additionally, the statistical properties are mainly discussed and studied based on the Bernoulli law of large numbers, and then a reliable macro feature is generated to describe the relationship between the statistical distribution and quality degradation of SCIs. Experimental results determined by using three public SCI databases show that the proposed method can perform better than relevant existing methods in the prediction of the visual quality of SCIs, especially in terms of the generalization for distortion type and interpretability for feature generation.
25 August 2021 in IEEE Transactions on Multimedia
Considering multidimensional structure of the multi-exposure images, a new Tensor product and Tensor-singular value decomposition based Multi-Exposure image Fusion (TT-MEF) method is proposed. The main innovation of this work is to explore a new feature representation of multi-exposure images in the new tensor domain and design the fusion strategy on this basis. Specifically, the luminance and the chrominance channels are fused separately to maintain color consistency. For the luminance fusion, the luminance channel of multi-exposure images is divided into two parts, that is, de-mean term and mean term. The de-mean term is represented as a tensor to extract the feature. Then, the tensor product and tensor-singular value decomposition (T-SVD) are used to design a tensor feature extractor. Furthermore, a fusion strategy of the de-mean term is presented according to the visual saliency model, and a fusion strategy of the mean term is defined by the local and the global visual weights to control counterpoise between the local and global luminance. For the chrominance fusion, a new fusion strategy is also designed by the tensor product and T-SVD, similar to the luminance fusion. Finally, the fused image is obtained by combining the luminance and chrominance fusion. Experimental results show that the proposed TT-MEF method generally outperforms the existing state-of-the-art in terms of subjective visual quality and objective evaluation.