Edge-deployed railway safety monitoring demands pixel-perfect semantic segmentation under extreme constraints: detection failures threaten operational safety, yet trackside devices must process diverse environmental conditions (fog, rain, nighttime) at over 30 FPS with under 50M parameters. Traditional CNNs lack long-range modeling capabilities for distant obstacle detection, while Transformer-based methods impose prohibitive quadratic computational complexity unsuitable for edge deployment. We present HybridSeg, a novel architecture that reformulates visual state space modeling as a controllable Markov Decision Process, enabling context-adaptive information propagation through reinforcement learning. Our approach integrates: (1) meta-learned state space dynamics via Proximal Policy Optimization, where learned policies dynamically adjust state transition parameters based on gradient feedback and hidden state statistics, the first application of RL-based parameter adaptation to visual state space models; (2) Structure-Aware Deformable Mamba blocks combining four-directional scanning with deformable spatial attention for irregular geometry handling; (3) cross-scale attention fusion across four pyramid levels with learnable inter-scale dependency modeling; (4) explicit multi-scale consistency constraints stabilizing training and improving generalization. Evaluated on 8,000 railway surveillance images spanning four environmental conditions, HybridSeg achieves 92.34 ± 0.25% mIoU and 97.82 ± 0.12% pixel accuracy at 38.52 FPS with 45.28M parameters–outperforming state-of-the-art CNN, Transformer, and Mamba methods by 1.61-3.16% in accuracy while delivering 2.31× faster inference than comparable approaches. The architecture demonstrates robust cross-domain transfer (89.53% CDR) and competitive performance on Cityscapes (85.73%), CamVid (87.25%), and ADE20K (48.53%), validating practical deployment for safety-critical edge applications.

Real-time semantic segmentation in railway safety monitoring presents significant computational challenges for edge-deployed vision systems, where accurate object detection directly impacts operational safety under strict resource constraints. Existing approaches struggle to achieve optimal trade-offs between segmentation accuracy and computational efficiency while maintaining robustness across varying environmental conditions. To address these challenges, we propose HybridSeg, an efficient multi-scale vision architecture that integrates Mamba-based sequence modeling with hierarchical feature fusion for resource-constrained railway monitoring applications. Our framework incorporates four key technical innovations: (1) Structure-Aware Preprocessing (SAP) that enhances input features through multi-scale structural analysis, (2) Structure-Aware Deformable Mamba (SADM) blocks enabling efficient long-range dependency capture via multi-directional scanning and deformable spatial attention, (3) Multi-Scale Feature Fusion (MSFF) with cross-scale attention for hierarchical feature integration, and (4) Cross-Scale Consistency (CSC) training that enforces multi-scale feature alignment. The decoder employs gated skip connections for adaptive feature combination across resolution levels. A comprehensive evaluation of railway surveillance datasets demonstrates superior performance, achieving 84.8±0.004% mIoU and 93.8±0.003% pixel accuracy while maintaining real-time efficiency at 31.8 FPS with only 38.9M parameters. The architecture exhibits robust performance across diverse environmental conditions, showing merely 4.1 percentage points of degradation under challenging nighttime scenarios. Our method effectively segments three critical object classes — personnel, foreign objects, and railway infrastructure — delivering substantial improvements in safety-critical detection tasks while enabling practical deployment on resource-limited edge computing platforms.

Edge-deployed railway safety monitoring demands pixel-perfect semantic segmentation under extreme constraints: detection failures threaten operational safety, yet trackside devices must process diverse environmental conditions (fog, rain, nighttime) at over 30 FPS with under 50M parameters. Traditional CNNs lack long-range modeling capabilities for distant obstacle detection, while Transformer-based methods impose prohibitive quadratic computational complexity unsuitable for edge deployment. We present HybridSeg, a novel architecture that reformulates visual state space modeling as a controllable Markov Decision Process, enabling context-adaptive information propagation through reinforcement learning. Our approach integrates: (1) meta-learned state space dynamics via Proximal Policy Optimization, where learned policies dynamically adjust state transition parameters based on gradient feedback and hidden state statistics, the first application of RL-based parameter adaptation to visual state space models; (2) Structure-Aware Deformable Mamba blocks combining four-directional scanning with deformable spatial attention for irregular geometry handling; (3) cross-scale attention fusion across four pyramid levels with learnable inter-scale dependency modeling; (4) explicit multi-scale consistency constraints stabilizing training and improving generalization. Evaluated on 8,000 railway surveillance images spanning four environmental conditions, HybridSeg achieves 92.34$\pm$0.25\% mIoU and 97.82$\pm$0.12\% pixel accuracy at 38.52 FPS with 45.28M parameters—outperforming state-of-the-art CNN, Transformer, and Mamba methods by 1.61-3.16\% in accuracy while delivering 2.31$\times$ faster inference than comparable approaches. The architecture demonstrates robust cross-domain transfer (89.53\% CDR) and competitive performance on Cityscapes (85.73\%), CamVid (87.25\%), and ADE20K (48.53\%), validating practical deployment for safety-critical edge applications.