When navigating ships in cold regions, sea ice concentration plays a crucial role in determining a ship’s navigability. However, automatically extracting the sea ice concentration and floe size distribution remains challenging, due to the difficulty in detecting all the ice floes from the images captured in complex polar environments, particularly those that include both ships and sea ice. In this paper, we propose using the YOLACT network to address this issue. Cameras installed on the ship collect images during transit and an image dataset is constructed to train a model that can intelligently identify all the targets in the image and remove any noisy targets. To overcome the challenge of identifying seemingly connected ice floes, the non-maximum suppression (NMS) in YOLACT is improved. Binarization is then applied to process the detection results, with the aim of obtaining an accurate sea ice concentration. We present a color map and histogram of the associated floe size distribution based on the ice size. The speed of calculating the sea ice density of each image reaches 21 FPS and the results show that sea ice concentration and floe size distribution can be accurately measured. We provide a case study to demonstrate the effectiveness of the proposed approach.