Fiber-reinforced composites, such as glass fiber-reinforced polymer (GFRP), are widely used across industries but are susceptible to inclusion defects during manufacturing. Detecting and classifying these defects is crucial for ensuring material integrity. This study classifies four common inclusion defects—metal, peel ply, release paper, and PTFE film—in GFRP using terahertz technology and machine learning. Two GFRP sheets with inclusion defects at different depths were fabricated. Terahertz time-domain signals were acquired, and a cross-correlation-based deconvolution algorithm extracted impulse responses. LSTM-RNN, Bi-LSTM RNN, and 1D-CNN models were trained and tested on time-domain, frequency-domain, and impulse response signals. The defect-free region exhibited the highest classification accuracy. Bi-LSTM RNN achieved the best recall and macro F1-score, followed by 1D-CNN, while LSTM-RNN performed worse. Training with impulse response signals improved classification while maintaining accuracy.