This paper proposes a lightweight model combined with data augmentation for vehicle detection in an intelligent sensor system. Vehicle detection can be considered as a binary classification problem, vehicle or non-vehicle. Deep neural networks have shown high accuracy in audio classification, and convolution neural networks are widely used for audio feature extraction and audio classification. However, the performance of deep neural networks is highly dependent on the availability of large quantities of training data. Recordings such as tracked vehicles are limited, and data augmentation techniques can be applied to improve the overall detection accuracy. In our case, spectrogram augmentation is applied on the mel spectrogram before extracting the Mel-scale Frequency Cepstral Coefficients (MFCC) features to improve the robustness of the system. Then depthwise separable convolution is applied to the CNN network for model compression and migrated to the hardware platform of the intelligent sensor system. The proposed approach is evaluated on a dataset recorded in the field using intelligent sensor systems with microphones. The final frame-level accuracy achieved was 94.64% for the test recordings and 34% of the parameters were reduced after compression.

This paper presents a method for detecting the acoustic signals of moving vehicle in a field environment using a convolutional recurrent neural network and multichannel audio. First, to improve the discrimination ability of the proposed method, a sound event detection framework is applied to detect the moving vehicle, apart from other seven sound event classes which are common in the field environment. Second, the asymmetric convolution block structure is introduced in the convolution stage of the framework to improve the ability of the convolutional layer to capture the temporal derivative and the characteristics of the feature map in the frequency domain. Then, a channel time-frequency attention module is employed after the convolutional stage to enable the network to prioritize critical information and ignore details irrelevant to the feature map detection task. Finally, the proposed method is evaluated on a dataset generated from audio clips covering different recording situations in the field environment with a total length of 48 hours. Ablation studies are performed to demonstrate the effectiveness of the asymmetric convolution block and the channel time-frequency attention module. Comparison with existing methods shows that the proposed method can achieve better performance by showing an improvement of F1-score by 8.9%, AUC by 0.0372, and a reduction of error rate by 5.3%.

Recent deep learning-based face recognition (FR) methods have demonstrated remarkable performance in high-resolution (HR) or down-sampled low-resolution (LR) tasks. However, these methods often exhibit disappointing speed-accuracy tradeoffs when deployed on real-world LR scenarios due to limited model generalization. To this end, we propose a lightweight Face Transformer framework for real-world LRFR named LLRFaceFormer. First, we propose a Transformers-as-Convolutions (TaC) network using a Transformer layer to replace the matrix multiplication in the standard convolutional process. This hybrid approach combines the strengths of Transformers and CNNs, allowing the TaC network to extract sufficient effective identity information via a global receptive field while adaptively discarding redundant homogeneous identity information on constructed LR faces. Second, we propose a Transformer-specific adaptive average procedure that incorporates tensor shape operations and a depthwise (DW) convolution. This procedure enables the LLRFaceFormer framework to focus on different regions of the input images. We also introduce an identity-aware simulator that generates real-world-like blurred LR scenarios during the training process to reduce the distribution discrepancy between the training LR faces and the testing real-world LR faces. The identity-aware simulator is simultaneously trained with the FR network with a cooperative training strategy. Further experiments illustrate the significantly superior speed-accuracy tradeoffs over existing LRFR methods with state-of-the-art (SOTA) performance on several LRFR benchmarks.

Infrared and visible image fusion (IVIF) is a key technique for synthesizing a comprehensive representation of a scene by exploiting diverse perceptual information. However, existing fusion methods encounter challenges in simultaneously preserving intricate texture details and extracting high-level semantic information–essential for downstream vision tasks. To address these issues, this article presents B2MFuse, a novel bi-branch multi-scale IVIF network based on joint semantics injection. The bi-branch consists of an interactive detail branch (ID-branch) and a parallel semantic branch (PS-branch), both featuring dual paths for infrared and visible modality. The former employs a channel exchange strategy that maximises the capture of modality-specific details while obtaining complementary features from the alternate modality. The latter efficiently captures semantic information and provides flexible scene knowledge guidance to the ID-branch, facilitating the subsequent top-tobottom multi-scale feature fusion and reconstruction process. A Spatial Weighted Channel Attention Fusion Module (SWCAFM) is then meticulously designed to enhance the integration of crucial fine-grained features across different scales. Furthermore, a scene-perception loss function is tailored to account for variations in the original image content. The synergy between B2MFuse’s advanced architecture and loss function ensures robust and superior fusion results in diverse environments, in particular enhancing human visual observation and supporting downstream visual tasks. Extensive evaluations on four public datasets demonstrate the superiority of our B2MFuse, compared to the state-of-the-art IVIF methods. The source code is available at https://github.com/arkymeng/B2MFuse.

Creating realistic, fully animatable whole-body avatars from a single portrait is challenging due to limitations in capturing subtle expressions, body movements, and dynamic backgrounds. Current evaluation datasets and metrics fall short in addressing these complexities. To bridge this gap, we introduce the Whole-Body Benchmark Dataset (WB-DH), an open-source, multi-modal benchmark designed for evaluating whole-body animatable avatar generation. Key features include: (1) detailed multi-modal annotations for fine-grained guidance, (2) a versatile evaluation framework, and (3) public access to the dataset and tools at https://github.com/deepreasonings/WholeBodyBenchmark.