We consider weakly-supervised video anomaly detection in this work. This task aims to learn to localize video frames containing anomaly events with only binary video-level annotation, i.e. , anomaly vs. normal. Traditional approaches usually formulate it as a multiple instance learning problem, which ignore the intrinsic data imbalance issue that positive samples are very scarce compared to negative ones. In this paper, we focus on addressing this issue to boost detection performance further. We develop a new light-weight anomaly detection model that fully utilizes enough normal videos to train a classifier with a good discriminative ability for normal videos, and we employ it to improve the selectivity for anomalous segments and filter out normal segments. Specifically, in addition to boosting anomalous prediction, a novel contrastive attention module additionally produces a converted normal feature from anomalous video to refined anomalous predictions by maximizing the classifier making a mistake. Moreover, to remove the stubborn normal segments selected by the attention module, we also design an attention consistency loss to employ the classifier with high confidence for normal features to guide the attention module. Extensive experiments on two large-scale datasets, UCF-Crime, ShanghaiTech and XD-Violence, clearly demonstrate that our model largely improves frame-level AUC over the state-of-the-art. Code is released at https://github.com/changsn/Contrastive-Attention-for-Video-Anomaly-Detection .
17 June 2021 in IEEE Transactions on Image Processing
The Coarse-To-Fine (CTF) matching scheme has been widely applied to reduce computational complexity and matching ambiguity in stereo matching and optical flow tasks by converting image pairs into multi-scale representations and performing matching from coarse to fine levels. Despite its efficiency, it suffers from several weaknesses, such as tending to blur the edges and miss small structures like thin bars and holes. We find that the pixels of small structures and edges are often assigned with wrong disparity/flow in the upsampling process of the CTF framework, introducing errors to the fine levels and leading to such weaknesses. We observe that these wrong disparity/flow values can be avoided if we select the best-matched value among their neighborhood, which inspires us to propose a novel differentiable Neighbor-Search Upsampling (NSU) module. The NSU module first estimates the matching scores and then selects the best-matched disparity/flow for each pixel from its neighbors. It effectively preserves finer structure details by exploiting the information from the finer level while upsampling the disparity/flow. The proposed module can be a drop-in replacement of the naive upsampling in the CTF matching framework and allows the neural networks to be trained end-to-end. By integrating the proposed NSU module into a baseline CTF matching network, we design our Detail Preserving Coarse-To-Fine (DPCTF) matching network. Comprehensive experiments demonstrate that our DPCTF can boost performances for both stereo matching and optical flow tasks. Notably, our DPCTF achieves new state-of-the-art performances for both tasks - it outperforms the competitive baseline (Bi3D) by 28.8% (from 0.73 to 0.52) on EPE of the FlyingThings3D stereo dataset, and ranks first in KITTI flow 2012 benchmark. The code is available at https://github.com/Deng-Y/DPCTF.
Time-of-Flight (ToF) sensors and stereo vision systems are both widely used for capturing depth data. They have some complementary strengths and limitations, which have been exploited in prior research to produce more accurate depth maps by fusing data from the two sources. However, among these diverse data fusion approaches, none of them provides an end-to-end neural network solution. In this work, we propose the first end-to-end ToF and stereo data fusion network using the coarse-to-fine matching framework, where the prior of ToF depth is integrated into the stereo matching process by constraining the search range of stereo matching within an interval around the ToF camera depth measurement. We adopt a dynamic search range for each pixel according to an estimated ToF error map, which is more efficient and effective than a constant one when handling various errors. The ToF error map is estimated by the ToF error estimator branching out from the stereo matching network. Both ToF error estimation and stereo matching are performed in a joint framework, with the two tasks assisting each other mutually. We also propose an upsampling module to replace the naive bilinear upsampling in the coarse-to-fine stereo matching network, which reduces the error caused by the upsampling. The proposed deep network is trained end-to-end on synthetic datasets and generalizable to real-world datasets without further fine-tuning. Experimental results show that our fusion method achieves higher accuracy than either ToF or stereo alone, and outperforms state-of-the-art fusion methods on both synthetic and real data.
28 January 2025 in IEEE Transactions on Microwave Theory and Techniques
Blood pressure (BP) can be estimated contactlessly based on Doppler wrist pulse (DWP) detection. However, this technique faces the challenge of interferences from the respiratory and its harmonic components, which can limit the accuracy of the BP estimation. This article first presents an adaptive respiratory harmonic filtering (ARHF) technique, aiming to mitigate the interference of respiration on the wrist pulse signal. Also, a customized 24-GHz continuous wave radar system with high linearity and a high gain antenna is used for better DWP detection. Second, a novel radar-based BP estimation model based on multidimensional morphological features extraction is proposed for high-accuracy and high-robustness BP estimation. In a stationary BP estimation experiment with eight subjects over three consecutive days, the proposed method demonstrates an average mean error and standard deviation of 2.60 $\pm$ 7.89 mmHg and 1.01 $\pm$ 2.84 mmHg in the sitting posture for systolic and diastolic blood pressure (SBP and DBP), respectively, achieving the highest estimation accuracy among the recent contactless radar-based BP detection. Additionally, a specific BP fluctuation range experiment is conducted to continuously measure BP, including varying BP fluctuations across five stages. In the worst case scenario, the proposed method exhibits SBP and DBP errors of 2.50 $\pm$ 2.98 and 8.34 $\pm$ 9.86 mmHg, respectively, affirming the robustness of the proposed technique. The experimental findings highlight the feasibility and reliability of the DWP extraction and subsequent contactless BP estimation techniques in the healthcare domain.