This paper presents a high performance vision-based system with a single static camera for traffic surveillance, for moving vehicle detection with occlusion handling, tracking, counting, and One Class Support Vector Machine (OC-SVM) classification. In this approach, moving objects are first segmented from the background using the adaptive Gaussian Mixture Model (GMM). After that, several geometric features are extracted, such as vehicle area, height, width, centroid, and bounding box. As occlusion is present, an algorithm was implemented to reduce it. The tracking is performed with adaptive Kalman filter. Finally, the selected geometric features: estimated area, height, and width are used by different classifiers in order to sort vehicles into three classes: small, midsize, and large. Extensive experimental results in eight real traffic videos with more than 4000 ground truth vehicles have shown that the improved system can run in real time under an occlusion index of 0.312 and classify vehicles with a global detection rate or recall, precision, and F-measure of up to 98.190%, and an F-measure of up to 99.051% for midsize vehicles.

Lane detection for traffic surveillance in intelligent transportation systems is a challenge for vision-based systems. In this paper, a novel pixel-entropy based algorithm for the automatic detection of the number of lanes and their centers, as well as the formation of their division lines is proposed. Using as input a video from a static camera, each pixel behavior in the gray color space is modeled by a time series; then, for a time period $\tau$ , its histogram followed by its entropy are calculated. Three different types of theoretical pixel-entropy behaviors can be distinguished: (1) the pixel-entropy at the lane center shows a high value; (2) the pixel-entropy at the lane division line shows a low value; and (3) a pixel not belonging to the road has an entropy value close to zero. From the road video, several small rectangle areas are captured, each with only a few full rows of pixels. For each pixel of these areas, the entropy is calculated, then for each area or row an entropy curve is produced, which, when smoothed, has as many local maxima as lanes and one more local minima than lane division lines. For the purpose of testing, several real traffic scenarios under different weather conditions with other moving objects were used. However, these background objects, which are out of road, were filtered out. Our algorithm, compared to others based on trajectories of vehicles, shows the following advantages: (1) the lowest computational time for lane detection (only 32 s with a traffic flow of one vehicle/s per-lane); and (2) better results under high traffic flow with congestion and vehicle occlusion. Instead of detecting road markings, it forms lane-dividing lines. Here, the entropies of Shannon and Tsallis were used, but the entropy of Tsallis for a selected q of a finite set achieved the best results.

Since multi-view learning leverages complementary information from multiple feature sets to improve model performance, a tensor-based data fusion layer for neural networks, called Multi-View Data Tensor Fusion (MV-DTF), is used. It fuses M feature spaces ${\mathcal{X}}_1,\cdots ,{\mathcal{X}}_M$, referred to as views, in a new latent tensor space, $\mathcal{S}$, of order P and dimension $J_1\times \cdots \times J_P$, defined in the space of affine mappings composed of a multilinear map $\mathcal{T}:{\mathcal{X}}_1\times \cdots \times {\mathcal{X}}_M\to \mathcal{S}$—represented as the Einstein product between a $(P+M)$-order tensor $\mathcal{A}$ anda rank-one tensor, $\mathcal{X}={\mathbf{x}}^{(1)}\otimes \cdots \otimes {\mathbf{x}}^{(M)}$, where ${\mathbf{x}}^{(m)}\in {\mathcal{X}}_m$ is the m-th view—and a translation. Unfortunately, as the number of views increases, the number of parameters that determine the MV-DTF layer grows exponentially, and consequently, so does its computational complexity. To address this issue, we enforce low-rank constraints on certain subtensors of tensor $\mathcal{A}$ using canonical polyadic decomposition, from which M other tensors ${\mathcal{U}}^{(1)},\cdots ,{\mathcal{U}}^{(M)}$, called here Hadamard factor tensors, are obtained. We found that the Einstein product $\mathcal{A}{⊛}_M\mathcal{X}$ can be approximated using a sum of R Hadamard products of M Einstein products encoded as ${\mathcal{U}}^{(m)}{⊛}_1{\mathbf{x}}^{(m)}$, where R is related to the decomposition rank of subtensors of $\mathcal{A}$. For this relationship, the lower the rank values, the more computationally efficient the approximation. To the best of our knowledge, this relationship has not previously been reported in the literature. As a case study, we present a multitask model of vehicle traffic surveillance for occlusion detection and vehicle-size classification tasks, with a low-rank MV-DTF layer, achieving up to $92.81\%$ and $95.10\%$ in the normalized weighted Matthews correlation coefficient metric in individual tasks, representing a significant $6\%$ and $7\%$ improvement compared to the single-task single-view models.