Worldwide, the access to safe drinking water is a huge problem. In fact, the number of persons without safe drinking water is increasing, even though it is an essential ingredient for human health and development. The enormity of the problem also makes it a critical environmental and public health issue. Therefore, there is a critical need for easy-to-use, compact and sensitive techniques for water quality monitoring. Raman spectroscopy has been a very powerful technique to characterize chemical composition and has been applied to many areas, including chemistry, food, material science or pharmaceuticals. The development of advanced Raman techniques and improvements in instrumentation, has significantly improved the performance of modern Raman spectrometers so that it can now be used for detection of low concentrations of chemicals such as in-line monitoring of chemical and pharmaceutical contaminants in water. This paper briefly introduces the fundamentals of Raman spectroscopy, reviews the development of Raman instrumentations and discusses advanced and potential Raman techniques for in-line water quality monitoring.