In this paper, we address the problem of video-based rain streak removal by developing an event-aware multi-patch progressive neural network. Rain streaks in video exhibit correlations in both temporal and spatial dimensions. Existing methods have difficulties in modeling the characteristics. Based on the observation, we propose to develop a module encoding events from neuromorphic cameras to facilitate deraining. Events are captured asynchronously at pixel-level only when intensity changes by a margin exceeding a certain threshold. Due to this property, events contain considerable information about moving objects including rain streaks passing though the camera across adjacent frames. Thus we suggest that utilizing it properly facilitates deraining performance non-trivially. In addition, we develop a multi-patch progressive neural network. The multi-patch manner enables various receptive fields by partitioning patches and the progressive learning in different patch levels makes the model emphasize each patch level to a different extent. Extensive experiments show that our method guided by events outperforms the state-of-the-art methods by a large margin in synthetic and real-world datasets.

To address the problems in the existing video super-resolution methods, such as noise, over smooth and visual artifacts, which are caused by reliance on limited external training or mismatch of internal similarity instances, this study proposes a video super-resolution reconstruction algorithm based on deep learning and spatio-temporal feature similarity (DLSS-VSR). The video super-resolution reconstruction mechanism with joint internal and external constraints is established utilizing both external deep correlation mapping learning and internal spatio-temporal nonlocal self-similarity prior constraint. A deep learning model based on deep convolutional neural network is constructed to learn the nonlinear correlation mapping between low-resolution and high-resolution video frame patches. A spatio-temporal feature similarity calculation method is proposed, which considers both internal video spatio-temporal self-similarity and external clean nonlocal similarity. For the internal spatio-temporal feature self-similarity, we improve the accuracy and robustness of similarity matching by proposing a similarity measure strategy based on spatio-temporal moment feature similarity and structural similarity. The external nonlocal similarity prior constraint is learned by patch group-based Gaussian mixture model. The time efficiency for spatio-temporal similarity matching is further improved based on saliency detection and region correlation judgment strategy. Experimental results demonstrate that the DLSS-VSR achieves competitive super-resolution quality compared to other state-of-the-art algorithms.

Food categorization is pivotal in numerous aspects of everyday life, assisting in the selection of food, managing diets, and addressing essential survival requirements. By leveraging the complementary information of various views, multi-view learning usually achieves superior performance compared to the single-view learning methods. However, characterized by the unrestrained openness of internet platforms and potential inconsistencies in food data collection processes, multi-view features often suffer from data loss, resulting in incomplete multi-view food data. Conventional multi-view clustering methods often falter in effectively capitalizing on the diverse correlations contained in food data, and exhibit limitations in dealing with the noise and irregularities pervading different views. Addressing these challenges, this paper presents the Robust Multi-Graph Contrastive network (RMGC) for multi-view food clustering. RMGC artfully combines multi-view representation learning with multi-graph contrastive regularization, creating a cohesive framework to manage incomplete multi-view data. By developing a multi-view encoding network, RMGC seamlessly blends various views into a cohesive representation, astutely assessing the significance of each view. More importantly, the proposed robust multi-graph contrastive regularization enhances the precision of the learned representation and successfully counteracts the noise and unreliability in multi-view data. The experiments conducted across several multi-view datasets manifest the effectiveness of RMGC, showing its superiority over existing methods. Our method not only making an advancement in food categorization but also contributes to the broader field of multi-view learning, offering innovative solutions for handling incomplete and noisy multi-view data.

Deep cross-media hashing provides an efficient semantic representation learning solution for large-scale cross-media retrieval. The existing methods only consider the inter-media or intra-media semantic association learning, ignore the guiding of semantic structure information, and have weak reasoning ability for implicit fine-grained semantic associations. To tackle this problem, we propose a novel structures aware fine-grained contrastive adversarial hashing method for cross-media retrieval. A novel cross-media contrastive adversarial hash network is constructed for the first time, which integrates the cross-media and intra-media contrastive learning and multi-modal adversarial learning, aiming at maximizing the semantic association between different modalities, and improving the semantic discrimination and consistency of cross-media unified hash representation, thereby the inter-media and intra-media semantic preserving ability can be well enhanced; A fine-grained cross-media semantic feature learning method based on fine-grained semantic reasoning with transformers is proposed, which captures fine-grained salient features of different modalities for semantic association learning, and enhances the reasoning ability of fine-grained implicit semantic association; A semantic label graph convolutional network guided cross-media semantic association learning strategy is proposed, which makes full use of semantic structure information to enhance the learning ability of implicit cross-media semantic associations. Extensive experiments on several large-scale cross-media benchmark datasets demonstrate that the proposed method outperforms the state-of-the-art methods.