The current audio single-mode self-supervised classification mainly adopts a strategy based on audio spectrum reconstruction. Overall, its self-supervised approach is relatively single and cannot fully mine key semantic information in the time and frequency domains. In this regard, this article proposes a self-supervised method combined with knowledge distillation to further improve the performance of audio classification tasks. Firstly, considering the particularity of the two-dimensional audio spectrum, both self-supervised strategy construction is carried out in a single dimension in the time and frequency domains, and self-supervised construction is carried out in the joint dimension of time and frequency. Effectively learn audio spectrum details and key discriminative information through information reconstruction, comparative learning, and other methods. Secondly, in terms of feature self-supervision, two learning strategies for teacher-student models are constructed, which are internal to the model and based on knowledge distillation. Fitting the teacher’s model feature expression ability, further enhances the generalization of audio classification. Comparative experiments were conducted using the AudioSet dataset, ESC50 dataset, and VGGSound dataset. The results showed that the algorithm proposed in this paper has a 0.5% to 1.3% improvement in recognition accuracy compared to the optimal method based on audio single mode.

With the advancements in multimedia software and hardware technology, image forgery localization has become an important challenge in digital forensics. To improve the efficiency and stability of image forgery detection, we propose a mixed-domain perception and multi-expert decoding recognition model. First, we design an alignment strategy that utilizes both RGB and frequency domain information of images. This strategy adapts to the multi-dimensional distribution characteristics of the original data, enhancing the discrimination of tampered regions. Next, we employ a hybrid expert modeling approach to improve the model’s robustness in the representation space through feature selection and recombination. Additionally, we introduce a region-weighted contrastive learning method to better localize and focus on tampered regions. Experiments on four datasets (CASIA, NIST, COVERAGE, and IMD) show that our proposed model achieves an improvement in AUC ranging from 0.15 to 1.9% compared to the existing advanced methods. These results indicate that our approach contributes to more accurate image forgery localization, offering potential benefits for digital forensics and multimedia security applications.

With the explosive growth of Internet video data, demands for accurate large-scale video classification and management are increasing. In the real-world deployment, the balance between effectiveness and timeliness should be fully considered. Generally, the video classification algorithm equipped with time segment network is used in industrial deployment, and the frame extraction feature is used to classify video actions However, the issue of semantic deviation will be raised due to coarse feature description. In this paper, we propose a novel method, called image dense feature and internal significant detail description, to enhance the generalization and discrimination of feature description. Specifically, the location information layer of space-time geometric relationship is added to effectively engrave the local features of convolution layer. Moreover, the multimodal feature graph network is introduced to effectively improve the generalization ability of feature fusion. Extensive experiments show that the proposed method can effectively improve the results on two commonly used benchmarks (kinetics 400 and kinetics 600).

Given that video content expression involves the combination of auditory and visual. How to effectively combine visual and audio features to guide video classification is still an exploratory problem. In this paper, we propose a Visual-Audio Cross-Attention Video Classification Algorithm (VACA-VCA). The method mainly includes the following aspects. 1) Tnspired by ViLT, the model calculation is focused on modality interaction at the feature fusion level to improve classification performance, and the dual-stream modeling is changed to concatenated single-stream modeling without introducing additional parameters. 2) To improve the generalization performance, a Visual-Audio Cross-Attention mechanism is proposed. In the experimental results on public datasets, the proposed method achieved significant improvements in the Kinetics 400 and Kinetics 600 video classification datasets.