Railway intrusion seriously threatens railway safety and can cause enormous loss of life and property. Therefore, railway intrusion detection is crucial for the safety of railway operation. Among the current methods of intrusion detection, machine vision-based methods have been widely used in railways, and have attracted close attention because of their great benefits. This paper proposes a comprehensive review of railway intrusion detection based on machine vision, covering ground monitoring, on-board inspection, and unmanned aerial vehicle (UAV) inspection. First, this paper systematically reviews most of the studies over the past two decades and presents the survey in three parts. Second, by analyzing these studies and the requirements for railway monitoring, we summarize the major challenges that hinder railway intrusion detection based on machine vision. Finally, we propose several promising perspectives for railway intrusion detection based on machine vision by comprehensively considering the development of machine vision, sensors, and pattern recognition together with the needs of railway scenes.

The operation of trains is substantially jeopardized by the presence of railway obstacles. Recent research on vision-based railway obstacle detectors has emphasized using sizeable neural network models to pursue a high accuracy while neglecting power consumption and training costs. To accelerate the model updating, training, and deployment cycle applied in the railway safety guarantee system, this article proposes a lightweight three-stage detection framework to identify obstacles in the single railway image. It is mainly comprised of the coarse region proposal (CRP) module, the lightweight railway obstacle detection network (RODNet), and postprocessing stage. In the CRP module, we set the binarized normed gradient (BING) detector to work on two different scale images for a more effective generation of proposal regions. Then, the intersection-over-minimum (IoM) metric is presented to filter and merge generated subregions. In the RODNet, a more lightweight and effective detection model is designed based on substantial improvements of YOLOv4-tiny, to simultaneously predict bounding boxes on the whole image and in each subregion. The RODNet’s backbone is rebuilt with one-shot aggregation and attention mechanism to reduce parameters and to maintain feature extraction abilities. Moreover, some residual blocks are also integrated into RODNet’s neck and head to improve the detection performance. Finally, a series of operations is devised to fuse, remove, and suppress predicted boxes on the boundary of each candidate subregion to reduce false-positive results at the postprocessing stage. Experimental results in the railway obstacle dataset show that RODNet reduces 48% parameters and 29% floating-point operations per second (FLOPs) with at least a 68.2% improvement of inference speed in the CPU compared with YOLOv4-tiny. Meanwhile, the proposed framework provides 80.3 mean average precision (mAP) at a speed of 65.9 ms/pic in the low-power graphics processing unit (GPU) and achieves a competitive or even better performance compared with the state-of-the-art large detection models.

Efficient perimeter intrusion detection approach is vital to safety management of China high-speed railway. In this article, for high-speed railway perimeter intrusion detection problem, a perimeter intrusion detection framework based on Internet of Things is designed to integrate different detection sensors. A data fusion algorithm was proposed to fuse multisensor data for improving detection precision. Computational results of normal condition and severe weather environment show our approach is effective and efficient for high-speed railway perimeter intrusion detection under Internet of Things, and is significant for safety management of China high-speed railway.

With the increase in railway operation mileage, railway perimeter security is facing significant challenges. These challenges are especially pronounced in low-light conditions such as dusk, night-time, or tunnels, where edge details are often lost due to insufficient light. Conventional detection networks rely heavily on precise boundary guidance of targets, resulting in a high rate of false alarms and omissions during on-site detection. To this end, this study proposes a target boundary dynamic perception detection network (BDP-Net) under railway low-light environments. Firstly, by constructing a spatial feature enhancement module, the low-light image is enhanced in the whole domain to strengthen the feature difference between the target and the background. Secondly, a parallel spatial enhancement convolutional two-branch network structure is used to retain the spatial information and generate the high-resolution features, and the two-branch network features are fused across the layers. Finally, a boundary dynamic representation module is proposed to process the fused feature information to enhance the information richness of the detected targets and improve the flexibility of regression. Extensive experiments on the RailDark and ExDark datasets demonstrate that the proposed BDP-Net achieves a 6.9% accuracy improvement on RailDark (reaching 80.1%) and a 2.1% accuracy improvement on ExDark (reaching 83.3%) compared to other detection networks (e.g., PEYOLO). The proposed algorithm can improve the accuracy and real-time performance of railway monitoring in low-light conditions, helping to prevent safety hazards and reduce accidents.

Trains shuttle in semiopen environments, and the surrounding environment plays an important role in the safety of train operation. The weather is one of the factors that affect the surrounding environment of railways. Under haze conditions, railway monitoring and staff vision could be blurred, threatening railway safety. This paper tackles image dehazing for railways. The contributions of this paper for railway video image dehazing are as follows: (1) this paper proposes an end-to-end residual block-based haze removal method that consists of two subnetworks, namely fine-grained and coarse-grained network can directly generate the clean image from input hazy image, called RID-Net (Railway Image Dehazing Network). (2) The combined loss function (per-pixel loss and perceptual loss functions) is proposed to achieve both low-level features and high-level features so to generate the high-quality restored images. (3) We take the full-reference criterion (PSNR&SSIM), object detection, running time, and sensory vision to evaluate the proposed dehazing method. Experimental results on railway synthesized dataset, benchmark indoor dataset, and real-world dataset demonstrate our method has superior performance compared to the state-of-the-art methods.