Calibrating the parameters of an evolutionary algorithm (EA) is a laborious task. The highly stochastic nature of an EA typically leads to a high variance of the measurements. The standard statistical method to reduce variance is measurement replication, i.e., averaging over several test runs with identical parameter settings. The computational cost of measurement replication scales with the variance and is often too high to allow for results of statistical significance. In this paper we study an alternative: the REVAC method for Relevance Estimation and Value Calibration, and we investigate how different levels of measurement replication influence the cost and quality of its calibration results. Two sets ofof experiments are reported: calibrating a genetic algorithm on standard benchmark problems, and calibrating a complex simulation in evolutionary agent-based economics. We find that measurement replication is not essential to REVAC, which emerges as a strong and efficient alternative to existing statistical methods.

Calibrating an evolutionary algorithm (EA) means finding the right values of algorithm parameters for a given problem. This issue is highly relevant, because it has a high impact (the performance of EAs does depend on appropriate parameter values), and it occurs frequently (parameter values must be set before all EA runs). This issue is also highly challenging, because finding good parameter values is a difficult task. In this paper we propose an algorithmic approach to EA calibration by describing a method, called REVAC, that can determine good parameter values in an automated manner on any given problem instance. We validate this method by comparing it with the conventional hand-based calibration and another algorithmic approach based on the classical meta-GA. Comparative experiments on a set of randomly generated problem instances with various levels of multi-modality show that GAs calibrated with REVAC can outperform those calibrated by hand and by the meta-GA.