In recent times, we have seen a massive rise in vision-based applications, such as video anomaly detection, motion detection, object tracking, people counting, etc. Most of these tasks are well defined, with a clear idea of the goal, along with proper datasets and evaluation procedures. However, perimeter intrusion detection (PID), which is one of the major tasks in visual surveillance, still needs to be formally defined. A perimeter intrusion detection system (PIDS) aims to detect the presence of an unauthorized object in a protected outdoor site during a certain time. Existing works vaguely define a PIDS, and this has a direct impact on the evaluation of methods. In this paper, we mathematically define it. We review the existing methods, datasets and evaluation protocols based on this definition. Furthermore, we provide a suitable evaluation protocol for real-life application. Finally, we evaluate the existing systems on available datasets using different evaluation schemes and metrics.