Data-driven analysis methods, such as the information content of a fitness sequence, characterize a discrete fitness landscape by quantifying its smoothness, ruggedness, or neutrality. However, enhancements to the information content method are required when dealing with continuous fitness landscapes. One typically employed adaptation is to sample the fitness landscape using random walks with variable step size. However, this adaptation has significant limitations: random walks may produce biased samples, and uncertainty is added because the distance between observations is not accounted for. In this paper, we introduce a robust information content-based method for continuous fitness landscapes, which addresses these limitations. Our method generates four measures related to the landscape features. Numerical simulations are used to evaluate the efficacy of the proposed method. We calculate the Pearson correlation coefficient between the new measures and other well-known exploratory landscape analysis measures. Significant differences on the measures between benchmark functions are subsequently identified. We then demonstrate the practical relevance of the new measures using them as class predictors on a machine learning model, which classifies the benchmark functions into five groups. Classification accuracy greater than 90% was obtained, with computational costs bounded between 1% and 10% of the maximum function evaluation budget. The results demonstrate that our method provides relevant information, at a low cost in terms of function evaluations.

Instance Space Analysis (ISA) is a recently developed methodology to (a) support objective testing of algorithms and (b) assess the diversity of test instances. Representing test instances as feature vectors, the ISA methodology extends Rice’s 1976 Algorithm Selection Problem framework to enable visualization of the entire space of possible test instances, and gain insights into how algorithm performance is affected by instance properties. Rather than reporting algorithm performance on average across a chosen set of test problems, as is standard practice, the ISA methodology offers a more nuanced understanding of the unique strengths and weaknesses of algorithms across different regions of the instance space that may otherwise be hidden on average. It also facilitates objective assessment of any bias in the chosen test instances and provides guidance about the adequacy of benchmark test suites. This article is a comprehensive tutorial on the ISA methodology that has been evolving over several years, and includes details of all algorithms and software tools that are enabling its worldwide adoption in many disciplines. A case study comparing algorithms for university timetabling is presented to illustrate the methodology and tools.

This article presents a method to generate diverse and challenging new test instances for continuous black-box optimization. Each instance is represented as a feature vector of exploratory landscape analysis measures. By projecting the features into a two-dimensional instance space, the location of existing test instances can be visualized, and their similarities and differences revealed. New instances are generated through genetic programming which evolves functions with controllable characteristics. Convergence to selected target points in the instance space is used to drive the evolutionary process, such that the new instances span the entire space more comprehensively. We demonstrate the method by generating two-dimensional functions to visualize its success, and ten-dimensional functions to test its scalability. We show that the method can recreate existing test functions when target points are co-located with existing functions, and can generate new functions with entirely different characteristics when target points are located in empty regions of the instance space. Moreover, we test the effectiveness of three state-of-the-art algorithms on the new set of instances. The results demonstrate that the new set is not only more diverse than a well-known benchmark set, but also more challenging for the tested algorithms. Hence, the method opens up a new avenue for developing test instances with controllable characteristics, necessary to expose the strengths and weaknesses of algorithms, and drive algorithm development.