This paper presents a new superpixel-based hand gesture recognition system based on a novel superpixel earth mover's distance metric, together with Kinect depth camera. The depth and skeleton information from Kinect are effectively utilized to produce markerless hand extraction. The hand shapes, corresponding textures and depths are represented in the form of superpixels, which effectively retain the overall shapes and color of the gestures to be recognized. Based on this representation, a novel distance metric, superpixel earth mover's distance (SP-EMD), is proposed to measure the dissimilarity between the hand gestures. This measurement is not only robust to distortion and articulation, but also invariant to scaling, translation and rotation with proper preprocessing. The effectiveness of the proposed distance metric and recognition algorithm are illustrated by extensive experiments with our own gesture dataset as well as two other public datasets. Simulation results show that the proposed system is able to achieve high mean accuracy and fast recognition speed. Its superiority is further demonstrated by comparisons with other conventional techniques and two real-life applications.

Object detection and tracking are two fundamental tasks in multicamera surveillance. This paper proposes a framework for achieving these tasks in a nonoverlapping multiple camera network. A new object detection algorithm using mean shift (MS) segmentation is introduced, and occluded objects are further separated with the help of depth information derived from stereo vision. The detected objects are then tracked by a new object tracking algorithm using a novel Bayesian Kalman filter with simplified Gaussian mixture (BKF-SGM). It employs a Gaussian mixture (GM) representation of the state and noise densities and a novel direct density simplifying algorithm for avoiding the exponential complexity growth of conventional Kalman filters (KFs) using GM. When coupled with an improved MS tracker, a new BKF-SGM with improved MS algorithm with more robust tracking performance is obtained. Furthermore, a nontraining-based object recognition algorithm is employed to support object tracking over nonoverlapping network. Experimental results show that: 1) the proposed object detection algorithm yields improved segmentation results over conventional object detection methods and 2) the proposed tracking algorithm can successfully handle complex scenarios with good performance and low arithmetic complexity. Moreover, the performance of both nontraining- and training-based object recognition algorithms can be improved using our detection and tracking results as input.

Weakly Supervised Anomaly Detection (WSAD) in surveillance videos is a complex task since usually only video-level annotations are available. Previous work treated it as a regression problem by giving different scores on normal and anomaly events. However, the widely used mini-batch training strategy may suffer from the data imbalance between these two types of events, which limits the model's performance. In this work, a cross-epoch learning (XEL) strategy associated with a hard instance bank (HIB) is proposed to introduce additional information from previous training epochs. Two new losses are proposed for XEL to achieve a higher detection rate as well as a lower false alarm rate of anomaly events. Moreover, the proposed XEL can be directly integrated into any existing WSAD framework. Experimental results of three XEL embedded models have shown promising AUC improvement (3%∼7%) on two public datasets, surpassing the state-of-the-art methods. Our code is available at: https://github.com/sdjsngs/XEL-WSAD .

Unsupervised video anomaly detection (UVAD) has gained significant attention due to its label-free nature. Typically, UVAD methods can be categorized into two branches, i.e. the one-class classification (OCC) methods and fully UVAD ones. However, the former may suffer from data imbalance and high false alarm rates, while the latter relies heavily on feature representation and pseudo-labels. In this paper, a novel feature reconstruction and disruption model (FRD-UVAD) is proposed for effective feature refinement and better pseudo-label generation in fully UVAD, based on cascade cross-attention transformers, a latent anomaly memory bank and an auxiliary scorer. The clip features are reconstructed using the space-time intra-clip information, as well as cross-inter-clip knowledge. Moreover, instead of blindly reconstructing all training features as OCC methods, a new disruption process is proposed to cooperate with the feature reconstruction simultaneously. Using the collected pseudo anomaly samples, it is able to emphasize the feature differences between normal and abnormal events. Additionally, a pre-trained UVAD scorer is utilized as a different criteria for anomaly prediction, which further refines the pseudo-labels. To demonstrate its effectiveness, comprehensive experiments and detailed ablation studies are conducted on three video benchmarks, namely CUHK Avenue, ShanghaiTech and UCF-Crime. Our proposed model (FRD-UVAD) achieves the best AUC performance (91.23%, 80.14%, and 82.12%) on all three datasets, surpassing other state-of-the-art OCC and fully UVAD methods. Furthermore, it obtains the lowest false alarm rate with a lower scene dependency, compared with other OCC methods. The code is available at https://github.com/tcc-power/FRD-unsupervised-video-anomaly-detection .

Training-based spectral reconstruction from RGB images is an efficient, inexpensive technique to recover spectral images using trichromatic cameras. Existing methods handle training samples individually without any consideration of local spatial and local spectral correlations between samples, which result in high metamerism and inaccurate reconstruction. In this letter, we propose for the first time the concept of spectral image reconstruction from RGB images with both chromatic and texture priors. We reduce redundancy of the sample set by applying a volume maximization based selection strategy. Taking advantage of the local linearity and sparsity properties of spectra in dictionary learning, we propose a locally linear embedded sparse reconstruction method taking into account both RGB values of pixels and the features of patch texture. Experimental results show our method is significantly more accurate than the current state-of-the-art methods.

Hyperspectral images (HSIs) provide rich spectral information that has been widely used in numerous computer vision tasks. However, their low spatial resolution often prevents their use in applications such as image segmentation and recognition. Fusing low-resolution HSIs with high-resolution RGB images to reconstruct high-resolution HSIs has attracted great research attention recently. In this paper, we propose an unsupervised blind fusion network that operates on a single HSI and RGB image pair and requires neither known degradation models nor any training data. Our method takes full advantage of an unrolling network and coordinate encoding to provide a state-of-the-art HSI reconstruction. It can also estimate the degradation parameters relatively accurately through the neural representation and implicit regularization of the degradation model. The experimental results demonstrate the effectiveness of our method both in simulations and in our real experiments. The proposed method outperforms other state-of-the-art nonblind and blind fusion methods on two popular HSI datasets. Our related code and data is available at https://github.com/CPREgroup/Real-Spec-RGB-Fusion .