Accurate thickness recognition plays a vital role in safeguarding the structural reliability of critical assets. Pulse eddy current testing (PECT), as a non-destructive method that is both non-contact and insensitive to surface coatings, provides an efficient pathway for this purpose. Nevertheless, the complex, nonstationary, and nonlinear characteristics of PECT signals make it difficult for conventional models to jointly capture localized high-frequency patterns and long-range temporal dependencies, thereby constraining their prediction performance. To overcome these issues, we introduce a novel deep learning framework, multi-scale residual dilated convolution, and bidirectional long short-term memory with a squeeze-and-excitation mechanism (MRDC-BiLSE) for PECT time series analysis. The architecture integrates a multi-scale residual dilated convolution block. By combining dilated convolutions with residual connections at different scales, this block captures structural patterns across multiple temporal resolutions, leading to more comprehensive and discriminative feature extraction. Furthermore, to better exploit temporal dependencies, the BiLSTM-SE module combines bidirectional modeling with a squeeze-and-excitation mechanism, resulting in more discriminative feature representations. Experiments on experimental PECT datasets confirm that MRDC-BiLSE surpasses existing methods, showing applicability for real-world thickness recognition.