30 January 2023 in IEEE Sensors Journal
Unsupervised deep-learning methods with a deep clustering model are widely used to detect anomaly human actions obtained by surveillance camera due to their powerful image feature learning abilities. These methods aim to optimize a model through a clustering induction target to provide useful cluster assignment, and usually use a forward propagation process: an autoencoder reconstructs an input sequence, a clustering model provides cluster allocation, and a scoring model evaluates distribution and provides scores for each sample. However, these methods have two main problems: one is that an autoencoder is difficult to handle human posture when abnormal and normal actions occur in crowd scenes captured by surveillance camera. The other is that network updating is interrupted by feature extraction and clustering in one epoch. To solve these problems, we design a deep memory storage clustering method based on graph convolution autoencoder to implement the real-time updating of pseudo-labels and network parameters. We also design a new loss function to express similarity between sample feature and centroid feature in a memory storage module and to constrain the parameter update of a network.We evaluate the unsupervised method for three important and representative datasets mainly composed of surveillance videos and use the AUC score as an experimental evaluation indicator.
31 December 2024 in IEEE Transactions on Intelligent Transportation Systems
In 3D intelligent transportation systems, effectively using multimodal data from cameras and LiDAR for object tracking is a key research area. Current methods struggle due to an overreliance on 3D LiDAR data that overlooks detailed 2D camera imagery and inefficient use of localization data, resulting in poor frame-by-frame object tracking. Therefore, we propose a Multimodal Localization Fusion Tracking (MLFT) method to overcome these problems in intelligent transportation systems. The proposed method enables precise localization of multimodal information inputs through a novel Multimodal Localization Distillation (MMLD) approach and achieves accurate frame-to-frame object tracking via an advanced Multimodal Knowledge Transform-Fusion Tracking (MKTF) technique. The proposed method is evaluated using three representative datasets: KITTI, NuScenes, and Waymo Open. The experimental results show that our proposed MLFT algorithm has strong performance, improving the success rate by about 3.0%, 4.1%, and 3.3% on the KITTI, NuScenes, and Waymo Open datasets, respectively, while improving the accuracy by about 3.2%, 3.1%, and 4.1%.