17 March 2022 in IEEE Transactions on Evolutionary Computation
Genetic programming (GP) is a common method for performing symbolic regression that relies on the use of ephemeral random constants in order to adequately scale predictions. Suitable values for these constants must be drawn from appropriate, but typically unknown, distributions for the problem being modelled. While rarely used with GP, Z-score standardisation of feature and response spaces often significantly improves the predictive performance of GP by removing scale issues and reducing error due to bias. However, in some cases it is also associated with erratic error due to variance. This paper demonstrates that this variance component increases in the presence of gaps at the boundaries of the training data explanatory variable intervals. An initial solution to this problem is proposed that augments training data with pseudo instances located at the boundaries of the intervals. When applied to benchmark problems, particularly with small training samples, this solution reduces error due to variance and therefore total error. Augmentation is shown to also stabilise error in larger problems, however results suggest that standardised GP works well on such problems with little need for training data augmentation.
23 February 2017 in Genetic Programming and Evolvable Machines
28 April 2020 in IEEE Transactions on Evolutionary Computation
An error function can be used to select between candidate models but it does not provide a thorough understanding of the behaviour of a model. A greater understanding of an algorithm can be obtained by performing a bias-variance decomposition. Splitting the error into bias and variance is effective for understanding a deterministic algorithm such as k-Nearest Neighbour, which provides the same predictions when performed multiple times using the same data. However, simply splitting the error into bias and variance is not sufficient for non-deterministic algorithms, such as genetic programming (GP), which potentially produces a different model each time it is run, even when using the same data. This paper presents an extended bias-variance decomposition that decomposes error into bias, external variance (error attributable to limited sampling of the problem) and internal variance (error due to random actions performed in the algorithm itself). This decomposition is applied to GP to expose the three components of error, providing a unique insight into the role of maximum tree depth, number of generations, size/complexity of function set and data standardisation in influencing predictive performance. The proposed tool can be used to inform targeted improvements for reducing specific components of model error.
3 September 2023 in Scientometrics
A common method for quantifying the interdisciplinarity of a publication is to measure the diversity of the publication’s cited references based on their disciplines. Here we examine the criteria that must be satisfied to develop a meaningful interdisciplinary measure based on citations and discuss the stages where uncertainty or bias may be introduced. In addition, using the Rao-Stirling diversity measure as an exemplar for such citation-based measures, we show how bootstrapping can be used to estimate a confidence interval for interdisciplinarity. Using an academic publication database, this approach is used to develop and assess a reliability measure for interdisciplinarity that extends current methods. Our results highlight issues with citation analysis for measuring interdisciplinarity and offer an approach to improve the confidence in assessing this concept. Specific guidelines for assessing the confidence in the Rao-Stirling diversity measure and subsequently other similar diversity measures are presented, hopefully reducing the likelihood of drawing false inferences about interdisciplinarity in the future.
1 December 2019 in RAE: Revista de Administracao de Empresas