The peer group of an image pixel is a pixel similarity-based concept which has been successfully used to devise image denoising methods. However, since it is difficult to define the pixel similarity in a crisp way, we propose to represent this similarity in fuzzy terms. In this paper, we introduce the fuzzy peer group concept, which extends the peer group concept in the fuzzy setting. A fuzzy peer group will be defined as a fuzzy set that takes a peer group as support set and where the membership degree of each peer group member will be given by its fuzzy similarity with respect to the pixel under processing. The fuzzy peer group of each image pixel will be determined by means of a novel fuzzy logic-based procedure. We use the fuzzy peer group concept to design a two-step color image filter cascading a fuzzy rule-based switching impulse noise filter by a fuzzy average filtering over the fuzzy peer group. Both steps use the same fuzzy peer group, which leads to computational savings. The proposed filter is able to efficiently suppress both Gaussian noise and impulse noise, as well as mixed Gaussian-impulse noise. Experimental results are provided to show that the proposed filter achieves a promising performance.