The aim of the violent recognition task is to determine whether a video contains violent behaviors. Given that violent behavior often comes with visual and audio anomalies, multimodal approaches have always played an important role in this field. However, existing methods have been limited by the insufficient utilization of audio-visual self-supervised semantic cues and correlation, resulting in a restricted representational capacity of the network and low generalization due to the scarcity of available violent video datasets. To address this issue, we propose a violent action recognition model based on global-local visual and audio contrastive learning. Our model introduces global and local contrastive objectives to achieve audio-visual multi-grained semantic alignment and leverage the correlation for violent video recognition. Experimental results demonstrate that our proposed model improves state-of-the-art by 2.31% on the VSD dataset, 0.71% on the Violent-Flows dataset, and 1.43% on the VCD dataset.

Video captioning aims to generate a grammatical and accurate sentence to describe a video. Recent methods have mainly tackled this problem by considering multiple modalities, yet they have neglected the difference in modalities and the importance of shrinking the gap between video and text. This paper proposes a multi-task video-captioning method with a Stepwise Multimodal Encoder. The encoder can flexibly digest multiple modalities by assigning a proper encoding depth for each modality. We also exploit both video-to-text (V2T) and text-to-video (T2V) flows by adding an auxiliary task of video–text semantic matching. We successfully achieve state-of-the-art performance on two widely known datasets: MSVD and MSR-VTT: (1) with the MSVD dataset, our method achieves an 18% improvement in CIDEr; (2) with the MSR-VTT dataset, our method achieves a 6% improvement in CIDEr.

Large-scale pretrained image-language models have shown remarkable performance recently. However, building a video-language model is more challenging due to the complexity of video and the difficulty of collecting high-quality data. This paper builds a video-language model in an adaptive manner, which transfers the knowledge from the image domain and can achieve state-of-the-art performance without any further massive video pretraining. The main contributions include a Visual Perception Adapter that seamlessly and efficiently adapts a pretrained image-language model to the video domain and a fine-grained contrastive learning with Inter-modal Token Alignment that bridges semantic gaps between vision, audio, and language with less data. The proposed model is evaluated on video captioning and retrieval. Experiments demonstrate that the proposed model exhibits competitive performance compared to models pretrained on millions of video-text pairs. Notably, our model's CIDEr and R@1 scores on the MSR-VTT dataset exceed the existing state-of-the-art by 6.3% and 1.3%.

Video Anomaly Understanding (VAU) extends traditional Video Anomaly Detection by enabling both identification and interpretation of abnormal events via natural language. Building on instruction-tuned multimodal large language models (MLLMs), we present a unified VAU framework and perform multi-task LoRA fine-tuning. We construct PreVAD-Instruct27k, a large-scale dataset of 27,854 videos annotated with categories, descriptions, and open-ended question answering (QA) pairs. PreVAD-Instruct27k enables joint optimization of classification, description, and QA tasks. Under sparse frame sampling, our model achieves 77.72% classification accuracy, 93.62% UniEval score for abnormal description, and 90.64% UniEval score for QA on PreVAD-Instruct27k, while maintaining strong generalization on XD-Violence and UCF-Crime. Ablation studies confirm the effectiveness of the multi-task design.