People with hearing impairments often face increased risks related to traffic accidents due to their reduced ability to perceive surrounding sounds. Given the cost and usage limitations of traditional hearing aids and cochlear implants, this study aims to develop a sound alert assistance system (SAAS) to enhance situational awareness and improve travel safety for people with hearing impairments. We proposed the VAS-Compass Net (Vehicle Alert Sound–Compass Net), which integrates three lightweight convolutional neural networks: EfficientNet-lite0, MobileNetV3-Small, and GhostNet. Through employing a fuzzy ranking ensemble technique, our proposed model can identify different categories of vehicle alert sounds and directions of sound sources on an edge computing device. The experimental dataset consisted of images derived from the sounds of approaching police cars, ambulances, fire trucks, and car horns from various directions. The audio signals were converted into spectrogram images and Mel-frequency cepstral coefficient images, and they were fused into a complete image using image stitching techniques. We successfully deployed our proposed model on a Raspberry Pi 5 microcomputer, paired with a customized smartwatch to realize an SAAS. Our experimental results demonstrated that VAS-Compass Net achieved an accuracy of 84.38% based on server-based computing and an accuracy of 83.01% based on edge computing. Our proposed SAAS has the potential to significantly enhance the situational awareness, alertness, and safety of people with hearing impairments on the road.

Recent multi-modal face anti-spoofing (FAS) methods have investigated the potential of leveraging multiple modalities to distinguish live and spoof faces. However, pre-adapted multi-modal FAS models often fail to detect unseen attacks from new target domains. Although a more realistic domain adaptation (DA) scenario has been proposed for single-modal FAS to learn specific spoof attacks during inference, DA remains unexplored in multi-modal FAS methods. In this paper, we propose a novel framework, MFAS-DANet, to address three major challenges in multi-modal FAS under the DA scenario: missing modalities, noisy pseudo labels, and model degradation. First, to tackle the issue of missing modalities, we propose extracting complementary features from other modalities to substitute missing modality features or enhance existing ones. Next, to reduce the impact of noisy pseudo labels during model adaptation, we propose deriving reliable pseudo labels by leveraging prediction uncertainty across different modalities. Finally, to prevent model degradation, we design an adaptive mechanism that decreases the loss weight during unstable adaptations and increasing it during stable ones. Extensive experiments demonstrate the effectiveness and state-of-the-art performance of our proposed MFAS-DANet.