Salt and pepper (SAP) noise elimination is a crucial step for further image processing and pattern recognition applications. The main aim of this paper is to propose a novel SAP noise elimination method which employs a regression based neuro-fuzzy network for highly corrupted gray scale and color images. In the proposed method, multiple neuro-fuzzy filters (NFF) trained with artificial bee colony (ABC) algorithm is combined with a decision tree (DT) algorithm. The performance of the proposed filter is compared with a number of well known methods with respect to popular metrics including, structural similarity index, peak signal to noise ratio and correlation on well-known test images. The results reveal that the proposed filter has superior performance in terms of all comparison metrics.