Oil spill pollution at sea causes significant damage to marine ecosystems. Quad-polarimetric Synthetic Aperture Radar (SAR) has become an essential technology since it can provide polarization features for marine oil spill detection. Using deep learning models based on polarimetric features, oil spill detection can be achieved. However, there is insufficient feature extraction due to model depth, small reception field lend due to loss of target information, and fixed hyperparameter for models. The effect of oil spill detection is still incomplete or misclassified. To solve the above problems, we propose an improved deep learning model named BO-DRNet. The model can obtain a more sufficiently and fuller feature by ResNet-18 as the backbone in encoder of DeepLabv3+, and Bayesian Optimization (BO) was used to optimize the model’s hyperparameters. Experiments were conducted based on ten prominent polarimetric features were extracted from three quad-polarimetric SAR images obtained by RADARSAT-2. Experimental results show that compared with other deep learning models, BO-DRNet performs best with a mean accuracy of 74.69% and a mean dice of 0.8551. This paper provides a valuable tool to manage upcoming disasters effectively.