In-process penetration monitoring of the pulsed laser welding process remains a great challenge for achieving uniform and reproducible products due to the highly complex nature of the keyhole dynamics within the intense laser-metal interactions. The main purpose of this study is to investigate the feasibility of acoustic emission (AE) measurement for penetration monitoring based on acoustic wave characteristics and deep learning. Firstly, a series of laser welding experiments on aluminum alloys were conducted using high-speed photography and AE techniques. This allowed us to in-situ visualize the complete keyhole dynamics and elucidate the generation mechanism of acoustic waves originating from pressure fluctuations at the keyhole wall. Then, an adaptive time-frequency technique namely VMD (Variational Mode Decomposition) was proposed to characterize the acoustic energy distribution among the nine subsignals with low-frequency and high-frequency components under different welding penetrations. Lastly, a novel hybrid model combing CNN (Convolutional Neural Network) and LSTM (Long Short Term Memory) was designed to deeply mine the spatial and temporal acoustic features from the extracted frequency components. Extensive experiments demonstrate that our proposed approach yields a remarkable classification performance with a test accuracy of 99.8% and a standard deviation of 0.21, which obtains a high recognition rate. This work is a new paradigm in the digitization and intelligence of the laser welding process and contributes to an alternative way of developing an efficient end-to-end penetration monitoring system.