Learning representations for multimedia content is critical for multimedia recommendation. Current representation learning methods roughly fall into two groups: (1) using the historical interactions to create ID embeddings of users and items, and (2) treating multi-modal data as the side information of items to enrich their ID embeddings. Each user-item interaction offers the supervisory signal to optimize the representation learning by the traditional supervised learning paradigm. Due to the overlook of the multi-modal patterns ($e.g.$, co-occurrence of visual, acoustic, textual features in micro-videos a user saw before, and her behavioral features) hidden in the data, these methods are insufficient to create powerful representations and obtain satisfactory recommendation accuracy. To capture multi-modal patterns in the data itself, we go beyond the supervised learning paradigm, and incorporate the idea of self-supervised learning (SSL) into multimedia recommendation. Specifically, SSL consists of two components: (1) data augmentation upon multi-modal contents, where we design three operators — feature dropout (FD), feature masking (FM), feature fine and coarse spaces (FAC) — to generate multiple views of individual items; and (2) contrastive learning, which differentiates the views of an item from the others’ to distill additional supervisory signals. Clearly, SSL enables us to explore and exhibit the underlying relations among modalities, thereby resulting in powerful representations. We denote the generic framework by Self-supervised Learning-guided Multimedia Recommendation (SLMRec). Extensive experiments are performed on three real-world datasets, showing that SLMRec achieves significant improvements over several state-of-the-art baselines like LightGCN [1], MMGCN [2]. Further analysis shows how SSL affects recommendation performance.

Transformer typically enjoys larger model capacity but higher computational loads than convolutional neural network (CNN) in vision tasks. In this letter, the advantages of such two networks are fused for achieving effective and efficient real image denoising. We propose a hybrid denoising model based on Transformer Encoder and Convolutional Decoder Network (TECDNet). The Transformer based on novel radial basis function (RBF) attention is used as encoder to improve the representation capability of overall model. In decoder, the residual CNN instead of Transformer is adopted to greatly reduce computational complexity of the whole denoising network. Extensive experimental results on real images show that TECDNet achieves the state-of-the-art denosing performance with relatively low computational cost.

As eating-out became an indispensable part of our daily lives, demand for the food recognition of unfamiliar restaurant increased significantly due to health-care. Although there are many researches on generic food recognition, there are relatively fewer studies on restaurant food image recognition. Meanwhile, it becomes extremely challenging for restaurant food image recognition due to insufficient food image. Prototypical network is common utilized to address the such a task in recent years. Although the methods based on prototypical network achieve impressive results in capturing similarities feature of the same food category, it fails to highlight important information on feature and instance level. Toward this end, we propose an effective food image recognition scheme by incorporating hybrid attention mechanism into prototypical network in this paper. Specifically, the image feature is first captured by convolutional neural network (CNN). Then the image attention weights yielded by instance-based attention mechanism are used to modulate the image feature of CNN for constructing class prototypes. And feature-based attention mechanism is employed to grasp important information of image for enriching image representation. Extensive experimental results on the large food image dataset verify that the performance of our proposed classification scheme outperforms the state-of-the-art ones.

Current image tampering localization methods primarily rely on Convolutional Neural Networks (CNNs) and Transformers. While CNNs suffer from limited local receptive fields, Transformers offer global context modeling at the expense of quadratic computational complexity. Recently, the state space model Mamba has emerged as a competitive alternative, enabling linear-complexity global dependency modeling. Inspired by it, we propose a lightweight and effective FORensic network based on vision MAmba (ForMa) for blind image tampering localization. Firstly, ForMa captures multi-scale global features that achieves efficient global dependency modeling through linear complexity. Then the pixel-wise localization map is generated by a lightweight decoder, which employs a parameter-free pixel shuffle layer for upsampling. Additionally, a noise-assisted decoding strategy is proposed to integrate complementary manipulation traces from tampered images, boosting decoder sensitivity to forgery cues. Experimental results on 10 standard datasets demonstrate that ForMa achieves state-of-the-art generalization ability and robustness, while maintaining the lowest computational complexity. Code is available at https://github.com/multimediaFor/ForMa.