This paper proposes a lightweight model combined with data augmentation for vehicle detection in an intelligent sensor system. Vehicle detection can be considered as a binary classification problem, vehicle or non-vehicle. Deep neural networks have shown high accuracy in audio classification, and convolution neural networks are widely used for audio feature extraction and audio classification. However, the performance of deep neural networks is highly dependent on the availability of large quantities of training data. Recordings such as tracked vehicles are limited, and data augmentation techniques can be applied to improve the overall detection accuracy. In our case, spectrogram augmentation is applied on the mel spectrogram before extracting the Mel-scale Frequency Cepstral Coefficients (MFCC) features to improve the robustness of the system. Then depthwise separable convolution is applied to the CNN network for model compression and migrated to the hardware platform of the intelligent sensor system. The proposed approach is evaluated on a dataset recorded in the field using intelligent sensor systems with microphones. The final frame-level accuracy achieved was 94.64% for the test recordings and 34% of the parameters were reduced after compression.

Recent deep learning-based face recognition (FR) methods have demonstrated remarkable performance in high-resolution (HR) or down-sampled low-resolution (LR) tasks. However, these methods often exhibit disappointing speed-accuracy tradeoffs when deployed on real-world LR scenarios due to limited model generalization. To this end, we propose a lightweight Face Transformer framework for real-world LRFR named LLRFaceFormer. First, we propose a Transformers-as-Convolutions (TaC) network using a Transformer layer to replace the matrix multiplication in the standard convolutional process. This hybrid approach combines the strengths of Transformers and CNNs, allowing the TaC network to extract sufficient effective identity information via a global receptive field while adaptively discarding redundant homogeneous identity information on constructed LR faces. Second, we propose a Transformer-specific adaptive average procedure that incorporates tensor shape operations and a depthwise (DW) convolution. This procedure enables the LLRFaceFormer framework to focus on different regions of the input images. We also introduce an identity-aware simulator that generates real-world-like blurred LR scenarios during the training process to reduce the distribution discrepancy between the training LR faces and the testing real-world LR faces. The identity-aware simulator is simultaneously trained with the FR network with a cooperative training strategy. Further experiments illustrate the significantly superior speed-accuracy tradeoffs over existing LRFR methods with state-of-the-art (SOTA) performance on several LRFR benchmarks.

This paper presents a method for detecting the acoustic signals of moving vehicle in a field environment using a convolutional recurrent neural network and multichannel audio. First, to improve the discrimination ability of the proposed method, a sound event detection framework is applied to detect the moving vehicle, apart from other seven sound event classes which are common in the field environment. Second, the asymmetric convolution block structure is introduced in the convolution stage of the framework to improve the ability of the convolutional layer to capture the temporal derivative and the characteristics of the feature map in the frequency domain. Then, a channel time-frequency attention module is employed after the convolutional stage to enable the network to prioritize critical information and ignore details irrelevant to the feature map detection task. Finally, the proposed method is evaluated on a dataset generated from audio clips covering different recording situations in the field environment with a total length of 48 hours. Ablation studies are performed to demonstrate the effectiveness of the asymmetric convolution block and the channel time-frequency attention module. Comparison with existing methods shows that the proposed method can achieve better performance by showing an improvement of F1-score by 8.9%, AUC by 0.0372, and a reduction of error rate by 5.3%.

Infrared and visible image fusion (IVIF) is a key technique for synthesizing a comprehensive representation of a scene by exploiting diverse perceptual information. However, existing fusion methods encounter challenges in simultaneously preserving intricate texture details and extracting high-level semantic information–essential for downstream vision tasks. To address these issues, this article presents B2MFuse, a novel bi-branch multi-scale IVIF network based on joint semantics injection. The bi-branch consists of an interactive detail branch (ID-branch) and a parallel semantic branch (PS-branch), both featuring dual paths for infrared and visible modality. The former employs a channel exchange strategy that maximises the capture of modality-specific details while obtaining complementary features from the alternate modality. The latter efficiently captures semantic information and provides flexible scene knowledge guidance to the ID-branch, facilitating the subsequent top-tobottom multi-scale feature fusion and reconstruction process. A Spatial Weighted Channel Attention Fusion Module (SWCAFM) is then meticulously designed to enhance the integration of crucial fine-grained features across different scales. Furthermore, a scene-perception loss function is tailored to account for variations in the original image content. The synergy between B2MFuse’s advanced architecture and loss function ensures robust and superior fusion results in diverse environments, in particular enhancing human visual observation and supporting downstream visual tasks. Extensive evaluations on four public datasets demonstrate the superiority of our B2MFuse, compared to the state-of-the-art IVIF methods. The source code is available at https://github.com/arkymeng/B2MFuse.

Currently, various studies have been exploring generation of long videos. However, the generated frames in these videos often exhibit jitter and noise. Therefore, in order to generate the videos without these noise, we propose a novel framework composed of four modules: separate tuning module, average fusion module, combined tuning module, and inter-frame consistency module. By applying our newly proposed modules subsequently, the consistency of the background and foreground in each video frames is optimized. Besides, the experimental results demonstrate that videos generated by our method exhibit a high quality in comparison of the state-of-the-art methods.

Comprehending occluded objects are not well studied in existing large-scale visual-language multi-modal models. Current state-of-the-art multi-modal large models struggles to provide satisfactory results in understanding occluded objects through universal visual encoders and supervised learning strategies. Therefore, we propose OCC-MLLM-CoT-Alpha, a multi-modal large vision language framework that integrates 3D-aware supervision and Chain-of-Thoughts guidance. Particularly, (1) we build a multi-modal large vision-language model framework which is consisted of a large multi-modal vision-language model and a 3D reconstruction expert model. (2) the corresponding multi-modal Chain-of-Thoughts is learned through a combination of supervised and reinforcement training strategies, allowing the multi-modal vision-language model to enhance the recognition ability with learned multi-modal chain-of-thoughts guidance. (3) A large-scale multi-modal chain-of-thoughts reasoning dataset, consisting of $110k$ samples of occluded objects held in hand, is built. In the evaluation, the proposed methods demonstrate decision score improvement of 15.75%,15.30%,16.98%,14.62%, and 4.42%,3.63%,6.94%,10.70% for two settings of a variety of state-of-the-art models.