In the modern era, the secure transmission and storage of information are among the utmost priorities. Optical security protocols have demonstrated significant advantages over digital counterparts, i.e., a high speed, a complex degree of freedom, physical parameters as keys (i.e., phase, wavelength, polarization, quantum properties of photons, multiplexing, etc.) and multi-dimension processing capabilities. This paper provides a comprehensive overview of optical cryptosystems developed over the years. We have also analyzed the trend in the growth of optical image encryption methods since their inception in 1995 based on the data collected from various literature libraries such as Google Scholar, IEEE Library and Science Direct Database. The security algorithms developed in the literature are focused on two major aspects, i.e., symmetric and asymmetric cryptosystems. A summary of state-of-the-art works is described based on these two aspects. Current challenges and future perspectives of the field are also discussed.