Haze reduces visibility, hindering real-time image processing applications. Although deep learning-based dehazing algorithms can significantly enhance image quality, their high computational complexity and storage demands make them challenging to deploy on resource-constrained hardware platforms. To address this, we propose an efficient dehazing neural network featuring physical perception model and cross-scale pixel attention (EP-CSANet), together with its hardware accelerator. In terms of algorithm, we design an efficient physical perception model (EPM) based on the atmospheric scattering model (ASM), combined with adaptive channel attention (ACA), which accurately approximates atmospheric light (A) and transmittance [t(x)], and effectively adapts to various environmental conditions, significantly improving dehazing accuracy. Furthermore, to enhance multiscale information interaction and preserve fine-grained texture details, we introduce cross-scale pixel attention (CSPA), which utilizes a dual-branch approach to extract high receptive-field information while maintaining fine-grained textures. In terms of hardware, we design a dedicated FPGA-based dehazing acceleration architecture. Through an efficient 16-stage pipeline, we achieve a dehazing rate of 127 frames per second (fps) for $640\times 480$ images, meeting real-time processing requirements. In addition, by incorporating sparse convolution optimization techniques, we significantly improve resource utilization: LUTs increased by 31.8%, FFs by 27.9%, and DSPs by 22.3%. Experimental results demonstrate that EP-CSANet, using only 2361 parameters on the SOTS dataset, outperforms other dehazing algorithms. The source code is publicly available at https://github.com/netflymachine/EP-CSANet