Recently, applying deep learning to no-reference image quality assessment (NR-IQA) has received significant attention. Especially in the last five years, an increasing interest has been drawn to the studies of rank learning since it can help mitigate the problem of small IQA datasets. However, on one hand, existing rank learning is not suitable for the authentically distorted images due to the lack of generated rank samples. On the other hand, the output of existing rank loss functions is uncontrollable, resulting in reduced performance. Motivated by these two limitations, we propose a novel rank learning based NR-IQA method, termed controllable list-wise ranking IQA (CLRIQA) in this paper. To be specific, we first present an imaging-heuristic approach, in which the over- and under-exposure is formulated as an inverse of the Weber-Fechner law, and fusion strategy and compression are adopted, to simulate the authentic distortion and generate the rank image samples. These samples are label-free yet associated with quality ranking information. Then we design a controllable list-wise ranking (CLR) loss function by setting an upper and lower bound of rank range and introducing an adaptive margin to tune rank interval. Finally, both the generated rank samples and proposed CLR are used to pre-train a convolutional neural network. Moreover, to obtain a more accurate prediction model, we take advantage of the IQA datasets to fine-tune the pre-trained network further. Various experiments are conducted on the IQA benchmark datasets, and experimental results demonstrate the effectiveness of the proposed CLRIQA method. The source code and network model can be downloaded at the following web address: https://github.com $/$ GZHU-DVL $/$ CLRIQA .

With the increasing maturity of the text-to-image and image-to-image generative models, AI-generated images (AGIs) have shown great application potential in advertisement, entertainment, education, social media, etc. Although remarkable advancements have been achieved in generative models, very few efforts have been paid to design relevant quality assessment models. In this paper, we propose a novel blind image quality assessment (IQA) network, named AMFF-Net, for AGIs. AMFF-Net evaluates AGI quality from three dimensions, i.e.“, visual quality”“, authenticity”, and “consistency”. Specifically, inspired by the characteristics of the human visual system and motivated by the observation that “visual quality” and “authenticity” are characterized by both local and global aspects, AMFF-Net scales the image up and down and takes the scaled images and original-sized image as the inputs to obtain multi-scale features. After that, an Adaptive Feature Fusion (AFF) block is used to adaptively fuse the multi-scale features with learnable weights. In addition, considering the correlation between the image and prompt, AMFF-Net compares the semantic features from text encoder and image encoder to evaluate the text-to-image alignment. We carry out extensive experiments on three AGI quality assessment databases, and the experimental results show that our AMFF-Net obtains better performance than nine state-of-the-art blind IQA methods. The results of ablation experiments further demonstrate the effectiveness of the proposed multi-scale input strategy and AFF block.

Recent studies have shown that deep learning-based classifiers are vulnerable to malicious inputs, i.e., adversarial examples. A practical solution is to construct a perceptible but localized perturbation called patch, making the well-trained models misclassified. However, most existing patch-based adversarial attacks focus on designing patches with localized rectangles, squares, or grids, ignoring the effect of the non-local patch. In this paper, we propose a novel cross-shaped patch attack paradigm (CSPA), a simple yet efficient and effective adversarial attack in Black-box scenarios. Specifically, the cross-shaped patch consists of two line segments intersected and perpendicular to each other at the midpoint. These two line segments are designed to be sufficiently thin and long to reach the four corners of the input image nearly. Thus, the patch has a globalized perturbation capacity while preserving its continuousness. The content and location of cross-shaped patch are then iteratively optimized by a carefully contrived random search-based algorithm to maximize this global property. Comprehensive experiments are conducted on four benchmark datasets against various victim networks. The results show that the proposed CSPA outperforms the existing patch-based attacks regarding both attack success rate and query efficiency by a large margin. Specifically, compared with the baselines, CSPA increases the success rate by up to 20% on ImageNet and reaches 100% on the CIFAR-100 and CIFAR-10 datasets. Meanwhile, CSPA reduces the average number of queries by up to 7 times. Even for the white-box attack scenario, our designed cross-shaped patch can still be applicable, achieving state-of-the-art performance.

Underwater image enhancement (UIE) is challenging since image degradation in aquatic environments is complicated and changing over time. Existing mainstream methods rely on either physical-model or data-driven, suffering from performance bottlenecks due to changes in imaging conditions or training instability. In this article, we attempt to adapt the diffusion model to the UIE task and propose a Content-Preserving Diffusion Model (CPDM) to address the above challenges. CPDM first leverages a diffusion model as its fundamental model for stable training and then designs a content-preserving framework to deal with changes in imaging conditions. Specifically, we construct a conditional input module by adopting both the raw image and the difference between the raw and noisy images as the input at each time step of the diffusion process, which can enhance the model’s adaptability by considering the changes involving the raw images in underwater environments. To preserve the essential content of the raw images, we construct a content compensation module for content-aware training by extracting low-level image features of the raw images as compensation for each down block. We conducted tests on the LSUI, UIEB, and EUVP datasets, and the results show that CPDM outperforms state-of-the-art methods in both subjective and objective metrics, achieving the best overall performance. The GitHub link for the code is https://github.com/GZHU-DVL/CPDM.

Approximate nearest neighbour search (ANNS) in high-dimensional space is an essential and fundamental operation in many applications from many domains such as multimedia database, information retrieval and computer vision. With the rapidly growing volume of data and the dramatically increasing demands of users, traditional heuristic-based ANNS solutions have been facing great challenges in terms of both efficiency and accuracy. Inspired by the recent successes of deep learning in many fields, substantial efforts have been devoted to applying deep learning techniques to ANNS for learning to index and learning to search, resulting in numerous algorithms that achieve state-of-the-art performance compared with conventional methods. In this survey paper, we comprehensively review the different types of deep learning-based ANNS methods according to two learning paradigms: learning to index and learning to search . We provide a comprehensive overview and analysis of these methods in a systematic manner. Based on the overview, we point out that end-to-end learning will be a new and promising research direction for deep learning-based ANNS, i.e., applying deep learning techniques to jointly learn the indexing and searching together, such that the underlying knowledge learned from data can directly contribute to the final searching performance. Finally, we conduct experiments and provide general performance analyses for the representative deep learning-based ANNS algorithms.