Infrared imaging has unique advantages in remote sensing observation and non-contact measurement, but its inherent low contrast and blurred structural details limit the reliability of scene interpretation by human observers. Unlike deep learning-based approaches that rely on data-driven training and substantial computational resources, we propose a Hybrid Prior Enhanced Decomposition (HPED) model, a training-free, model-driven framework that incorporates structural and luminance priors into a multi-stage enhancement pipeline. An l₁–l₀-regularized decomposition separates the input into a base layer that preserves global structures and salient edges and a detail layer in which low-amplitude fluctuations and noise are suppressed. A prior-preserving bi-gamma correction method enhances base-layer contrast through prior-guided histogram segmentation and adaptive gray-level redistribution. An improved grayscale mapping strategy further enhances global contrast while maintaining interframe consistency. Experiments on real SWIR, MWIR, and LWIR images show that HPED ranks first among evaluated traditional and deep learning-based methods on key perceptual quality metrics (SSIM, VIF, LIF), while achieving over 25 fps on a CPU-only platform, sufficient for smooth real-time visual display. Task-oriented evaluation further shows that the HPED improves CNR and SCR by 174.7 ± 11.4% and 298.5 ± 52.1% on average over the raw input, outperforming all competing methods and suggesting potential applicability in downstream machine perception tasks such as detection and tracking.