Recent global warming has resulted in shifting of weather patterns and led to intensification of natural disasters and upsurges in pests and diseases. As a result, global food systems are under pressure and need adjustments to meet the change—often by pesticides. Unfortunately, such agrochemicals are harmful for humans and the environment, and consequently need to be monitored. Traditional detection methods currently used are time consuming in terms of sample preparation, are high cost, and devices are typically not portable. Recently, Surface Enhanced Raman Scattering (SERS) has emerged as an attractive candidate for rapid, high sensitivity and high selectivity detection of contaminants relevant to the food industry and environmental monitoring. In this review, the principles of SERS as well as recent SERS substrate fabrication methods are first discussed. Following this, their development and applications for agrifood safety is reviewed, with focus on detection of dye molecules, melamine in food products, and the detection of different classes of pesticides such as organophosphate and neonicotinoids.