In global sensitivity analysis (GSA) of a model, a proper convergence analysis of metrics is essential for ensuring a level of confidence or trustworthiness in sensitivity results obtained, yet is somewhat deficient in practice. The level of confidence in sensitivity measures, particularly in relation to their influence and support for decisions from scientific, social and policy perspectives, is heavily reliant on the convergence of GSA. We review the literature and summarize the available methods for monitoring and assessing convergence of sensitivity measures based on application purposes. The aim is to expose the various choices for convergence assessment and encourage further testing of available methods to clarify their level of robustness. Furthermore, the review identifies a pressing need for comparative studies on convergence assessment methods to establish a clear hierarchy of effectiveness and encourages the adoption of systematic approaches for enhanced robustness in sensitivity analysis.
A computationally efficient and robust sampling scheme can support a sensitivity analysis of models to discover their behaviour through Quasi Monte Carlo approximation. This is especially useful for complex models, as often occur in environmental domains when model runtime can be prohibitive. The Sobol' sequence is one of the most used quasi-random low-discrepancy sequences as it can explore the parameter space significantly more evenly than pseudo-random sequences. The built-in determinism of the Sobol' sequence assists in achieving this attractive property. However, the Sobol' sequence tends to deteriorate in the sense that the estimated errors are distributed inconsistently across model parameters as the dimensions of a model increase. By testing multiple Sobol' sequence implementations, it is clear that the deterministic nature of the Sobol' sequence occasionally introduces relatively large errors in sensitivity indices produced by well-known global sensitivity analysis methods, and that the errors do not diminish by averaging through multiple replications. Problematic sensitivity indices may mistakenly guide modellers to make type I and II errors in trying to identify sensitive parameters, and this will potentially impact model reduction attempts based on these sensitivity measurements. This work investigates the cause of the Sobol' sequence's determinism-related issues. References I. A. Antonov and V. M. Saleev. An economic method of computing LPτ-sequences. USSR Comput. Math. Math. Phys. 19.1 (1979), pp. 252–256. doi: 10.1016/0041-5553(79)90085-5 P. Bratley and B. L. Fox. Algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 14.1 (1988), pp. 88–100. doi: 10.1145/42288.214372 J. Feinberg and H. P. Langtangen. Chaospy: An open source tool for designing methods of uncertainty quantification. J. Comput. Sci. 11 (2015), pp. 46–57. doi: 10.1016/j.jocs.2015.08.008 on p. C90). S. Joe and F. Y. Kuo. Constructing Sobol sequences with better two-dimensional projections. SIAM J. Sci. Comput. 30.5 (2008), pp. 2635–2654. doi: 10.1137/070709359 S. Joe and F. Y. Kuo. Remark on algorithm 659: Implementing Sobol’s quasirandom sequence generator. ACM Trans. Math. Soft. 29.1 (2003), pp. 49–57. doi: 10.1145/641876.641879 W. J. Morokoff and R. E. Caflisch. Quasi-random sequences and their discrepancies. SIAM J. Sci. Comput. 15.6 (1994), pp. 1251–1279. doi: 10.1137/0915077 X. Sun, B. Croke, S. Roberts, and A. Jakeman. Comparing methods of randomizing Sobol’ sequences for improving uncertainty of metrics in variance-based global sensitivity estimation. Reliab. Eng. Sys. Safety 210 (2021), p. 107499. doi: 10.1016/j.ress.2021.107499 S. Tarantola, W. Becker, and D. Zeitz. A comparison of two sampling methods for global sensitivity analysis. Comput. Phys. Com. 183.5 (2012), pp. 1061–1072. doi: 10.1016/j.cpc.2011.12.015 S. Tezuka. Discrepancy between QMC and RQMC, II. Uniform Dist. Theory 6.1 (2011), pp. 57–64. url: https://pcwww.liv.ac.uk/~karpenk/JournalUDT/vol06/no1/5Tezuka11-1.pdf I. M. Sobol′. On the distribution of points in a cube and the approximate evaluation of integrals. USSR Comput. Math. Math. Phys. 7.4 (1967), pp. 86–112. doi: 10.1016/0041-5553(67)90144-9 I. M. Sobol′. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp 1.4 (1993), pp. 407–414.