Decades of ongoing research have shown that background modelling is a very powerful technique, which is used in intelligent surveillance systems, in order to extract features of interest, known as foregrounds. In order to work with the dynamic nature of different scenes, many techniques of background modelling adopted the unsupervised approach of Gaussian Mixture Model with an iterative paradigm. Although the technique has had much success, a problem occurs in cases of sudden scene changes with high variation (e.g., illumination changes, camera jittering) that the model unknowingly and unnecessarily takes into account those effects and distorts the results. Therefore, this paper proposes an unsupervised, parallelized, and tensor-based approach that algorithmically works with entropy estimations. These entropy estimations are used in order to assess the uncertainty level of a constructed background, which predicts both the present and future variations from the inputs, thereby opting to use either the incoming frames to update the background or simply discard them. Our experiments suggest that this method is highly integrable into a surveillance system that consists of other functions and can be competitive with state-of-the-art methods in terms of processing speed.

The main goal of traffic surveillance systems (TSSs) is to extract useful traffic information by analyzing signals from cameras. This paper presents a system for vehicle detection and classification from static pole-mounted roadside surveillance cameras on busy streets in the presence of different kinds of vehicles. There has been considerable research to accommodate this subject since the 90s; but most studies have been only carried out in developed countries where traffic infrastructures are built around automobiles, whereas in developing countries, motorbikes are dominant. This paper proposes a method that robustly detects, classifies and counts vehicles into three classes: light (motorbikes, bikes, tricycles), medium (cars, sedans, SUV), heavy vehicle (trucks, buses), and a novel tracking algorithm designed to enable classification by majority voting to cope with motorbikes' sudden changes in direction. Extensive experiments with real-world data to evaluate the system's performance have shown promising results: a detection rate of 95.3% in daytime scenes.

Background Subtraction (BgS) is a problem for handling pixel-level identification of changing or moving entities in the field of view of a static camera system. Recent works have discovered the superior generalization to unseen realistic scenarios by an approach called deep BgS, which employs deep neural networks on concatenations of image inputs and their backgrounds. However, due to a lack of large-scale foreground-background image datasets, the challenges of manually creating background masks for existing Internet-scale image datasets, and a general lack of attention to circumvent them, studies to exploit the generalization potentials of deep BgS have yet been thoroughly conducted for unseen realistic scenes. Hence, our work proposes an analysis regarding the generalization capabilities of deep BgS, particularly by their binary constraints of pixel feature representations. Then, to address the lack of large-scaled labeled data to induce generalization, we propose a strategy to generate a virtually limitless freeform dataset, follow up with the first self-supervised pretraining strategy for deep BgS, then we extend it to a semi-supervised framework called BgSubNet, which can support inference on unseen scenarios by self-supervision and mixed-domain training with real and synthetic data. Our results are experimentally validated at competitive results with CAMO-UOW and CDnet2014 datasets to show considerably high performance against existing frameworks.

Shadows are among the most critical problems for traffic surveillance systems (TSSs). In a TSS, shadow regions significantly affect the extraction of vehicles’ attributes for vehicle detection, classification and tracking. Although many methods have been proposed to address this key problem, the dilemma of accurate shadow removal with vehicles’ boundaries recovery and real-time processing still poses as a great challenge. In this paper, we propose a new method for shadow removal that utilizes edge features to eliminate shadows, and to refine vehicles’ images regardless of the changes in illumination and shadow orientations. Experiments were done on real-world data to compare the results of our method with previous ones. Thorough investigation shows that our method gets rid of vehicles’ shadows more accurately and significantly restores conveyances’ images from shadow separation. In addition, our method is real-time.

Given Natural Language (NL) text descriptions, NL-based vehicle retrieval aims to extract target vehicles from a multi-view multi-camera traffic video pool. Due to inherent distinctions between textual and visual data, this is a challenging multi-modal retrieval task that requires robust feature extractors (e.g. neural network) to well-align the abstract representations of texts and images in the same domain. However, solutions to the problem have been challenged by the high data complexities of not only the multi-view, multi-camera attributes of visual data and the diverse range of textual descriptions but also a lack of high-volume datasets in this relatively new field, alongside a prominently large domain gap between training and test sets. Many existing approaches have developed computationally expensive models to separately extract the subspaces of language and vision before blending into the same shared representation space while only focusing on single-modal information and ignoring much of the multi-modal information to deal with the aforementioned issues. Hence, we propose a Domain Adaptive Knowledge-based Retrieval System (DAKRS) to effectively and efficiently align multi-modal knowledge in a setting of limited labels. Our contributions are threefold: (i) An efficient extension of Contrastive Language-Image Pre-training (CLIP)’s transfer learning into a baseline text-to-image multi-modular vehicle retrieval framework; (ii) A data enhancement module to create pseudo-vehicle tracks from the traffic video pool by leveraging the robustness of baseline retrieval model combine with background subtraction; and (iii) A SSDA (SSDA) scheme to engineer pseudo-labels for adapting model parameters to the target domain distribution. Experimental results are benchmarked on the Cityflow-NL dataset, illustrating our competitiveness against state-of-the-art performances in terms of effectiveness and efficiency without needing further post-processing or ensembling.