The current accuracy of speech recognition can reach over 97% on different datasets, but in noisy environments, it is greatly reduced. Improving speech recognition performance in noisy environments is a challenging task. Due to the fact that visual information is not affected by noise, researchers often use lip information to help to improve speech recognition performance. This is where the performance of lip recognition and the effect of cross-modal fusion are particularly important. In this paper, we try to improve the accuracy of speech recognition in noisy environments by improving the lip reading performance and the cross-modal fusion effect. First, due to the same lip possibly containing multiple meanings, we constructed a one-to-many mapping relationship model between lips and speech allowing for the lip reading model to consider which articulations are represented from the input lip movements. Audio representations are also preserved by modeling the inter-relationships between paired audiovisual representations. At the inference stage, the preserved audio representations could be extracted from memory by the learned inter-relationships using only video input. Second, a joint cross-fusion model using the attention mechanism could effectively exploit complementary intermodal relationships, and the model calculates cross-attention weights on the basis of the correlations between joint feature representations and individual modalities. Lastly, our proposed model achieved a 4.0% reduction in WER in a −15 dB SNR environment compared to the baseline method, and a 10.1% reduction in WER compared to speech recognition. The experimental results show that our method could achieve a significant improvement over speech recognition models in different noise environments.

Collaborative fraud has become increasingly serious in telecom and social networks, but is hard to detect by traditional fraud detection methods. In this paper, we find a significant positive correlation between the increase of collaborative fraud and the degraded detection performance of traditional techniques, implying that those fraudsters that are difficult to detect with traditional methods are often collaborative in their fraudulent behavior. As we know, multiple objects may contact a single target object over a period of time. We define multiple objects with the same contact target as generalized objects, and their social behaviors can be combined and processed as the social behaviors of one object. We propose Fraud Detection Model based on Second-order and Collaborative Relationship Mining (COFD), exploring new research avenues for collaborative fraud detection. Our code and data are released at https://github.com/CatScarf/COFD-MM https://github.com/CatScarf/COFD-MM.

The rapid development and popularization of video surveillance highlight the critical and challenging problem, vehicle reidentification, which suffers from the limited interinstance discrepancy between different vehicle identities and large intrainstance differences of the same vehicle. In this article, we propose a novel multilevel attention network to hierarchically learn an efficient feature embedding for vehicle re-ID. Three kinds of attention are designed in the network: hard local-level attention to localize vehicle salient parts, soft pixel-level attention to refine attended pixels both globally and locally, and spatial attention to enhance the encoder’s spatial awareness of salient regions within the windscreen area. Multigrain features are subsequently learned from semantic awareness to spatial awareness, guaranteeing the intraclass compactness and interclass separability for vehicle re-ID. Extensive experiments validate the effectiveness of each attention component and demonstrate that our approach outperforms the state-of-the-art re-ID methods on two challenging datasets: VehicleID and Vehicle-1 M.

Graph-based fraud detection (GFD) has garnered increasing attention due to its effectiveness in identifying fraudsters within multimedia data such as online transactions, product reviews, or telephone voices. However, the prevalent in-distribution (ID) assumption significantly impedes the generalization of GFD approaches to out-of-distribution (OOD) scenarios, which is a pervasive challenge considering the dynamic nature of fraudulent activities. In this paper, we introduce the Heterophilic Graph Invariant Learning Framework (HGIF), a novel approach to bolster the OOD generalization of GFD. HGIF addresses two pivotal challenges: creating diverse virtual training environments and adapting to varying target distributions. Leveraging edge-aware augmentation, HGIF efficiently generates multiple virtual training environments characterized by generalized heterophily distributions, thereby facilitating robust generalization against fraud graphs with diverse heterophily degrees. Moreover, HGIF employs a shared dual-channel encoder with heterophilic graph contrastive learning, enabling the model to acquire stable high-pass and low-pass node representations during training. During the Test-time Training phase, the shared dual-channel encoder is flexibly fine-tuned to adapt to the test distribution through graph contrastive learning. Extensive experiments showcase HGIF's superior performance over existing methods in OOD generalization, setting a new benchmark for GFD in OOD scenarios.

Under confrontational environment, individual can impersonate others by wearing masks or other skills to conceal real identity, which brings enormous challenges to physical identity recognition. Moreover, massive fake attributes seriously threaten identity management. Aiming at the existence of fake attributes during multimodal identification, we propose a novel method to detect fake attributes and compute real identity for security of systems. Most previous methods focused on the differences about features between fake and normal attributes, but each method generally targeted one type of fake attribute, and the results decreased a lot with unknown attacks. In this paper, we first explore the essential differences of data distribution caused by natural and human behaviors, then with order-of-consensus-calculation based on the differences, fake attributes are detected by analyzing the rank of consensus identity in recognition results, finally maximum-consensus-calculation is applied to compute real identity for evaluating detection performance. Experimental results on face, fingerprint, and voiceprint demonstrate that the proposed method can detect fake attributes effectively, which has a higher accuracy, and the accuracy of identity recognition is increased obviously by about 13.20% with forgery detection. The additional experiments further confirm the feasibility of the proposed method with increase of fake attributes. Furthermore, the proposed method can deal with different kinds of attacks, even with unknown attacks, and it is also significant to improve the security of identification with complex environment.