Due to the limitation of existing filters in detection and removal of fixed value impulse noise, the authors propose an iterative grouping median filter (IGMF) according to the characteristics of noise intensity and distribution. It sorts the noise‐free pixels in neighbourhood by intensity, divides the sorted pixels into groups depending on the intensity differences of adjacent pixels, and finally takes the median of the maximum group as the intensity of noisy pixel. This noise removal strategy is performed iteratively and takes full advantage of the previous denoising results. Experiments show that IGMF outperforms the existing state‐of‐the‐art filters in terms of visual perception, peak signal to noise ratio and structural similarity index at various noise densities.