The existing masked face recognition algorithms almost tend to adopt synthetic masked face datasets for training. However, these models are limited as they rely on existing mask augmentation methods, which contain few mask patterns and cannot simulate shadows and textures in realistic scenes. To overcome this limitation, we propose a semi-supervised face recognition framework to fully exploit unlabeled real masked face samples, improving the mask robustness of the recognition model. More specifically, unlike the original face embedding network, we design a part-aware network to explore multi-region face representation based on the face structure. In this way, we obtain multiple face sub-embeddings, which correspond to different regions of the face, including the upper half, the lower half and the whole. Crucially, we use the norm of the sub-embedding to represent the activation state of the facial region features. For the input unlabeled masked face image, we restrict the sub-embedding norm of its lower half to weaken the face feature representation of the occluded area. For normal face samples, their partial features are kept activated by maintaining the sub-embedding norm, which guides the deep network does not ignore the available information. Moreover, we employ the margin-based recognition loss for normal samples to ensure that the model is sufficiently discriminative for normal facial features. Extensive experimental results on both normal and real masked face datasets show that our approach significantly outperforms the state-of-the-arts. Code is available at https://github.com/Baojin-Huang/UFace.

No-reference image quality assessment (NR-IQA) methods, inspired by the free energy principle, improve the accuracy of image quality prediction by simulating the human brain’s repair process for distorted images. However, existing methods use separate optimization schemes for distortion restoration and quality prediction, which undermines the accurate mapping of feature representations to quality scores. To address this issue, we propose a joint restoration and quality feature learning NR-IQA (RQFL-IQA) method to jointly tackle distortion image restoration and quality prediction within a unified framework. To accurately establish the quality reconstruction relationship between distorted and restored images, a hybrid loss function based on pixel-wise and structure-wise representations is used to improve the restoration capability of the image restoration network. The proposed RQFL-IQA exploits rich labels, including restored images and quality scores, to enable the model to learn more discriminative features and establish a more accurate mapping from feature representation to quality scores. In addition, to avoid the impact of poor restoration on quality prediction, we propose a module with a cleaning function to reweight the fusion of restored and primitive features to achieve more perceptual consistency in feature fusion. Experimental results on public IQA datasets show that the proposed RQFL-IQA is superior over existing methods.

Image quality assessment (IQA) is an important problem in computer vision with many applications. We propose a transformer-based multi-task learning framework for the IQA task. Two subtasks: constructing an auxiliary information error map and completing image quality prediction, are jointly optimized using a shared feature extractor. We use visual transformers (ViT) as a feature extractor for feature extraction and guide ViT to focus on image quality-related features by building auxiliary information error map subtask. In particular, we propose a fusion network that includes a channel focus module. Unlike the fusion methods commonly used in previous IQA methods, we use the fusion network, including the channel attention module, to fuse the auxiliary information error map features with the image features, which facilitates the model to mine the image quality features for more accurate image quality assessment. And by jointly optimizing the two subtasks, ViT focuses more on extracting image quality features and building a more precise mapping from feature representation to quality score. With slight adjustments to the model, our approach can be used in both no-reference (NR) and full-reference (FR) IQA environments. We evaluate the proposed method in multiple IQA databases, showing better performance than state-of-the-art FR and NR IQA methods.