The objective of this study was to investigate the effect of ongoing use of an evaluation tool on hospitals’ emergency preparedness for mass casualty events (MCEs). Two cycles of evaluation of emergency preparedness were conducted based on measurable parameters. A significant increase was found in mean total scores between the 2 cycles (from 77.1 to 88.5). An increase was found in scores for standard operating procedures, training, and equipment, but the change was significant only in the training category. Relative increase was highest for hospitals that did not experience real MCEs. This study offers a structured and practical approach for ongoing improvement of emergency preparedness, based on validated, measurable benchmarks. Ongoing assessment of emergency preparedness motivates hospitals to improve capabilities and results in a more effective emergency response mechanism. Use of predetermined and measurable benchmarks allows the institutions being assessed to improve their level of performance in the areas evaluated.