Conformal coating optimization requires control of multiple application parameters to achieve target thickness and coverage uniformity, with spray atomization settings (atomization valve control) playing a crucial role in droplet formation and material deposition. Traditional manual tuning yields a low success rate and wastes significant materials through trial-and-error. We introduce Model-Enhanced Soft Actor-Critic (ME-SAC), a reinforcement learning approach that combines a hybrid dynamics model with uncertainty-aware planning. Our dynamics model integrates polynomial ridge regression with a residual neural ensemble, achieving R $$^{2}$$ = 0.70 using 5-fold cross-validation on the experimental dataset. Unlike conventional model-based RL that trains on synthetic data, ME-SAC uses the learned dynamics model for uncertainty-aware candidate-action evaluation without generating additional synthetic rollouts for replay-buffer augmentation. The RL agents are trained and evaluated through interactions with a surrogate coating environment constructed from 158 physical coating experiments. The planning module evaluates candidate actions using a conservative objective that penalizes high-uncertainty predictions, ensuring safe exploration. Experimental results demonstrate that ME-SAC achieves 95.0% success rate compared to 74.0% for model-free SAC baseline, reaching 90% performance using approximately 600 surrogate-environment interactions while baseline requires >1600 samples without achieving this threshold (>2.67 $$\times$$ sample efficiency). In the evaluated training runs, ME-SAC maintained performance without the mid-training degradation observed for the baseline.