Video salient object detection (VSOD) aims at locating the most attractive objects presented in video sequences by exploiting spatial and temporal cues. Previous methods mainly utilize convolutional neural networks (CNNs) to fuse or complement across RGB and optical flow cues via simple strategies. To take full advantage of CNNs and recently emerged Transformers, this letter proposes a novel mutual-guidance Transformer-embedding network, called MGT-Net, where a mutual-guidance multi-head attention mechanism (MGMA) explores more sophisticated long-range cross-modal interactions. Such a mechanism is designed into a new mutual-guidance Transformer (MGTrans) module that can propagate long-range contextual dependencies based on information of the other modality. To the best of our knowledge, MGT-Net is the first VSOD model that embeds Transformers as modules into CNNs for improved performance. Prior to MGTrans, we also propose and deploy a feature purification module (FPM) to purify noisy backbone features. Experimental results on five benchmark datasets demonstrate the state-of-the-art performance of MGT-Net.

Background clutters pose challenges to defocus blur detection. Existing approaches often produce artifact predictions in background areas with clutter and relatively low confident predictions in boundary areas. In this work, we tackle the above issues from two perspectives. Firstly, inspired by the recent success of self-attention mechanism, we introduce channel-wise and spatial-wise attention modules to attentively aggregate features at different channels and spatial locations to obtain more discriminative features. Secondly, we propose a generative adversarial training strategy to suppress spurious and low reliable predictions. This is achieved by utilizing a discriminator to identify predicted defocus map from ground-truth ones. As such, the defocus network (generator) needs to produce ‘realistic’ defocus map to minimize discriminator loss. We further demonstrate that the generative adversarial training allows exploiting additional unlabeled data to improve performance, a.k.a. semi-supervised learning, and we provide the first benchmark on semi-supervised defocus detection. Finally, we demonstrate that the existing evaluation metrics for defocus detection generally fail to quantify the robustness with respect to thresholding. For a fair and practical evaluation, we introduce an effective yet efficient $AUF_\beta $ metric. Extensive experiments on three public datasets verify the superiority of the proposed methods compared against state-of-the-art approaches.

Most existing image deblurring methods construct statistical prior to describe the difference between blur and clear image. They discard the position information and ignore pixel feature changing in deblurring, which results in inferior restoration performance for images unsatisfying corresponding assumptions. Intuitively, fuzziness of pixel belonging to different image regions will reduce along with image deblurring. This phenomenon could intrinsically describe the pixel characteristic. To this end, we analyze fuzziness of pixels and objects in a blurry image, and utilize the similarity between two fuzzy objects on image pixels to depict the blur degree of an image, which is inspired by overlap functions and overlap indices. To minimize the similarity between fuzzy objects, we introduce the non-parameters model to construct an integer programming problem. Energy minimization could significantly reduce the similarity between two fuzzy objects. Experimental results show that the proposed method can achieve better performance than the state-of-the-art blind deblurring methods on benchmark datasets and natural images.

Image ordinal classification has drawn substantial attention from the research community due to the ordering relation between image categories. Recent advancements towards image ordinal classification lie in applying deep neural networks [convolutional neural network (CNN)]. Nevertheless, the lack of ordinal training data prevents deep models from generalising to testing data. In this work, two multi‐view learning approaches are proposed to tackle the insufficient data issue. On one hand, a multi‐view ordinal classification with multi‐view max pooling (MVMP) approach is proposed, in which each image is randomly blocked with some grids thus creating multiple views of the original data. All views are then used to train multi‐view CNN for classification. On the other hand, in order to account for the ordinal relation, the authors propose a double‐task learning on MVMP for classification and average pooling for regression. The task of regression benefits that of classification, mainly focusing on improving classification's recognition accuracy. The two proposed approaches are validated on Adience dataset, and show very compelling results. The code and models will be available online.

Referring image segmentation aims to segment object in an image based on a referring expression. Its difficulty lies in aligning expression semantics with visual instances. The existing methods based on semantic reasoning are limited by the performance of external syntax parser and do not explicitly explore the relationships between visual instances. This article proposes an end-to-end method for referring image segmentation by aligning ’linguistic relationship’ with ’visual relationships’. This method does not rely on external syntax parser for expression parsing. In this paper, the expression is adaptively and structurally parsed into three components: ’subject’, ’object’, and ’linguistic relationship’ by the Semantic Component Parser (SCP) in a learnable manner. Instances Activation Map Module (IAM) locates multiple visual instances based on the subject and object. In addition, the Relationship Based Visual Localization Module (RBVL) firstly enables each instance of the image to learn global knowledge, then decodes the visual relationships between these visual instances, and finally aligns the visual relationships with the linguistic relationships to further accurately locate the target object. The experimental results show that the proposed method improves performance by 4– 9% compared with baseline method on multiple referring image segmentation datasets.