In order to realize the real-time detection of an unmanned fishing speedboat near a ship ahead, a perception platform based on a target visual detection system was established. By controlling the depth and width of the model to analyze and compare training, it was found that the 5S model had a fast detection speed but low accuracy, which was judged to be insufficient for detecting small targets. In this regard, this study improved the YOLOv5s algorithm, in which the initial frame of the target is re-clustered by K-means at the data input end, the receptive field area is expanded at the output end, and the loss function is optimized. The results show that the precision of the improved model’s detection for ship images was 98.0%, and the recall rate was 96.2%. Mean average precision (mAP) reached 98.6%, an increase of 4.4% compared to before the improvements, which shows that the improved model can realize the detection and identification of multiple types of ships, laying the foundation for subsequent path planning and automatic obstacle avoidance of unmanned ships.