The video-based separation of foreground (FG) and background (BG) has been widely studied due to its vital role in many applications, including intelligent transportation and video surveillance. Most of the existing algorithms are based on traditional computer vision techniques that perform pixel-level processing assuming that FG and BG possess distinct visual characteristics. Recently, state-of-the-art solutions exploit deep learning models targeted originally for image classification. Major drawbacks of such a strategy are the lacking delineation of FG regions due to missing temporal information as they segment the FG based on a single frame object detection strategy. To grapple with this issue, we excogitate a 3D convolutional neural network (3D CNN) with long short-term memory (LSTM) pipelines that harness seminal ideas, viz., fully convolutional networking, 3D transpose convolution, and residual feature flows. Thence, an FG-BG segmenter is implemented in an encoder-decoder fashion and trained on representative FG-BG segments. The model devises a strategy called double encoding and slow decoding, which fuses the learned spatio-temporal cues with appropriate feature maps both in the down-sampling and up-sampling paths for achieving well generalized FG object representation. Finally, from the Sigmoid confidence map generated by the 3D CNN-LSTM model, the FG is identified automatically by using Nobuyuki Otsu's method and an empirical global threshold. The analysis of experimental results via standard quantitative metrics on 16 benchmark datasets including both indoor and outdoor scenes validates that the proposed 3D CNN-LSTM achieves competitive performance in terms of figure of merit evaluated against prior and state-of-the-art methods. Besides, a failure analysis is conducted on 20 video sequences from the DAVIS 2016 dataset.
6 August 2021 in IEEE Transactions on Intelligent Transportation Systems
Defects on rail surfaces, which have become critical problems, need to be detected and removed as quickly as possible to ensure the fast, safe, and stable operation of trains. At present, although many solutions have been proposed to address these problems, the comprehensiveness, rapidity, and accuracy of defect detection remain unsatisfactory. This study aims to resolve these existing problems and accordingly proposes a multi-model rail surface defect detection system based on convolutional neural networks (MRSDI-CNN) from the standpoint of studying the squat on the rail surface. The convolutional neural networks utilized include the improved Single Shot MultiBox Detector (SSD) and You Only Look Once version 3(YOLOv3)—two types of one-stage networks. We expounded and analyzed the performance of the convolutional neural networks as well as their applicability to rail surface defect detection. We used a diverse range of rail defect sizes to improve the detection performance of the two deep learning networks, following which they could identify three types of squats in parallel with improved accuracy and without reduction of the detection speed. The experimental results confirm the effectiveness and superiority of the proposed method over those of previous studies.
22 August 2019 in IEEE Transactions on Vehicular Technology
The automatic detection of foreground (FG) objects in videos is a demanding area of computer vision, with essential applications in video-based traffic analysis and surveillance. New solutions have attempted exploiting deep neural network (DNN) for this purpose. In DNN, learning agents, i.e., features for video FG object segmentation is nontrivial, unlike image segmentation. It is a temporally processed decision-making problem, where the agents involved are the spatial and temporal correlations of the FG objects and the background (BG) of the scene. To handle this and to overcome the conventional DL models' poor delineation at the borders of FG regions due to fixed-view receptive filed-based learning, this work introduces a Multi-view Receptive Field Encoder-Decoder Convolutional Neural Network called MvRF-CNN.
Measuring fluid characteristics is of high importance in various industries such as the polymer, petroleum, and petrochemical industries, etc. Flow regime classification and void fraction measurement are essential for predicting the performance of many systems. The efficiency of multiphase flow meters strongly depends on the flow parameters. In this study, MCNP (Monte Carlo N-Particle) code was employed to simulate annular, stratified, and homogeneous regimes. In this approach, two detectors (NaI) were utilized to detect the emitted photons from a cesium-137 source. The registered signals of both detectors were decomposed using a discrete wavelet transform (DWT). Following this, the low-frequency (approximation) and high-frequency (detail) components of the signals were calculated. Finally, various features of the approximation signals were extracted, using the average value, kurtosis, standard deviation (STD), and root mean square (RMS). The extracted features were thoroughly analyzed to find those features which could classify the flow regimes and be utilized as the inputs to a network for improving the efficiency of flow meters. Two different networks were implemented for flow regime classification and void fraction prediction. In the current study, using the wavelet transform and feature extraction approach, the considered flow regimes were classified correctly, and the void fraction percentages were calculated with a mean relative error (MRE) of 0.4%. Although the system presented in this study is proposed for measuring the characteristics of petroleum fluids, it can be easily used for other types of fluids such as polymeric fluids.
25 October 2018 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Machine learning algorithms such as those for object classification in images, video content analysis (VCA), and human action recognition are used to extract meaningful information from data recorded by image sensors and cameras. Among the existing machine learning algorithms for such purposes, extreme learning machines (ELMs) and online sequential ELMs (OS-ELMs) are well known for their computational efficiency and performance when processing large datasets. The latter approach was derived from the ELM approach and optimized for real-time application. However, OS-ELM classifiers are computationally demanding, and the existing state-of-the-art computing platforms are not efficient enough for embedded systems, especially for applications with strict requirements in terms of low power consumption, high throughput, and low latency. This work presents the implementation of an ELM/OS-ELM in a customized system-on-a-chip field-programmable gate array (SoC FPGA)-based architecture to ensure efficient hardware acceleration. The acceleration process comprises parallel extraction, deep pipelining and efficient shared memory communication.
30 March 2020 in IEEE Transactions on Industrial Informatics
This paper presents a new supervised multilayer subnetwork-based feature refinement and classification model for representation learning. The novelties of this algorithm are as follows: 1) Different from most multi-layer networks that go deeper with increased number of network layers, this work architects a model with wider subnetwork nodes. 2) The conventional classification methods adopt a separate search mechanism to derive a generalized feature space and to get the final cognition; but this work proposes a one-shot process to find the meaningful latent space and recognize the objects. 3) The traditional feature representation and image classification approaches apply a unimodal feature coding, which suffers from incapable of global knowledge. This work overcomes the pitfall through multimodal fusion that fuses various feature sources into one super-state encoding to achieve higher performance. A cross-domain experimental study on camera identification and image classification shows that the proposed method achieves superior performance compared to the existing models.