In modern semiconductor manufacturing, optimizing plasma etching processes is key for advancing technology and achieving profitable production. In this study, we propose an efficient process flow to calibrate the etching mechanisms and optimize the etch profile by combining three-dimensional plasma simulations with machine learning (ML). Our proposed workflow speeds up the model calibration process and effectively addresses data scarcity issues by integrating extensive simulation data with TEM and other data sources. We have developed a three-dimensional Cl2/HBr/O2 plasma etching model for shallow trench isolation (STI), which is based on physical and chemical reaction mechanisms and external parameters consistent with actual production requirements, such as gas ratio, time duration, and chamber conditions. Machine learning is employed to tackle both forward and inverse problems. In the forward problem, the model predicts etching profiles under varying process conditions, with accuracy and similarity assessed using multiple criteria, including the structural similarity index (SSIM). Our approach achieves a notable reduction in prediction time, completing forecasts in 0.5 seconds compared to the 6500 CPU seconds required by TCAD. In the inverse problem, the model adjusts physical parameters to match observed results, improving its predictive accuracy. After model calibration, predictions are validated with wafer test outcomes, showing relative errors below 7%, thereby confirming the accuracy and robustness of the model parameters and the effectiveness of the entire process.

This study proposes a novel framework integrating long short-term memory (LSTM) networks with Bayesian optimization (BO) to address process–device co-optimization challenges in trench-gate metal–oxide–semiconductor field-effect transistor (MOSFET) manufacturing. Conventional TCAD simulations, while accurate, suffer from computational inefficiency in high-dimensional parameter spaces. To overcome this, an LSTM-based TCAD proxy model is developed, leveraging hierarchical temporal dependencies to predict electrical parameters (e.g., breakdown voltage, threshold voltage) with deviations below 3.5% compared to physical simulations. The model, validated on both N-type and P-type 20 V trench MOS devices, outperforms conventional RNN and GRU architectures, reducing average relative errors by 1.78% through its gated memory mechanism. A BO-driven inverse optimization methodology is further introduced to navigate trade-offs between conflicting objectives (e.g., minimizing on-resistance while maximizing breakdown voltage), achieving recipe predictions with a maximum deviation of 8.3% from experimental data. Validation via TCAD-simulated extrapolation tests and SEM metrology confirms the framework’s robustness under extended operating ranges (e.g., 0–40 V drain voltage) and dimensional tolerances within industrial specifications. The proposed approach establishes a scalable, data-driven paradigm for semiconductor manufacturing, effectively bridging TCAD simulations with production realities while minimizing empirical trial-and-error iterations.