The problem of image deraining has drawn widespread attention as the blurry images caused by rain streaks can severely impair the practical value of images. Although exiting deep learning-based methods have achieved significant success, single-image deraining still faces great challenges due to the diverse orientations, multi-scale distributions and intense texture entanglement of rain streaks. Specifically, existing state-space-based deraining methods still suffer from relatively uniform local enhancement across different feature hierarchies, representation bias caused by fixed single-direction scanning in two-dimensional (2D) images, and insufficient selection of cross-level information during feature fusion. In this paper, a novel deraining network named HLDA-Net is proposed to address these issues, which unifies hierarchical local enhancement (HLE) and direction-aware dual-branch state-space module (DDS) within a Convolutional Neural Network (CNN) encoder–decoder framework. Level-adaptive detail refinement is embedded in HLE for better background texture restoration, while complementary dual-branch spatial modeling is introduced in DDS to mitigate the directional bias of single-path scanning and strengthen long-distance dependency modeling. In addition, gated skip fusion (GSF) is integrated into the encoder–decoder restoration pipeline to strengthen cross-level feature aggregation and optimize stability. Extensive experiments on synthetic and real-world benchmarks demonstrate the effectiveness of HLDA-Net for single-image deraining, with the peak signal-to-noise ratio (PSNR) and Structural Similarity Index Measure (SSIM) on the Rain100H dataset reaching 31.034 and 0.914, respectively.