Virtual experiments are a digital representation of a real measurement and play a crucial role in modern measurement sciences and metrology. Beyond their common usage as a modeling and validation tool, a virtual experiment may also be employed to perform a parameter sensitivity analysis or to carry out a measurement uncertainty evaluation. For the latter to be compliant with statistical principles and metrological guidelines, the procedure to obtain an estimate and a corresponding measurement uncertainty requires careful consideration. We employ a Monte Carlo sampling procedure using a virtual experiment that allows one to perform a measurement uncertainty evaluation according to the Monte Carlo approach of JCGM-101 and JCGM-102, two widely applied guidelines for uncertainty evaluation in metrology. We extend and formalize a previously published approach for simple additive models to account for a large class of non-linear virtual experiments and measurement models for multidimensionality of the data and output quantities, and for the case of unknown variance of repeated measurements. With the algorithm developed here, a simple procedure for the evaluation of measurement uncertainty is provided that may be applied in various applications that admit a certain structure for their virtual experiment. Moreover, the measurement model commonly employed for uncertainty evaluation according to JCGM-101 and JCGM-102 is not required for this algorithm, and only evaluations of the virtual experiment are performed to obtain an estimate and an associated uncertainty of the measurand. We demonstrate the efficacy of the developed approach and the effect of the underlying assumptions for a generic polynomial regression example and an example of a simplified coordinate measuring machine and its virtual representation. The results of this work highlight that considerable effort, diligence, and statistical considerations need to be invested to make use of a virtual experiment for uncertainty evaluation in a way that ensures equivalence with the accepted guidelines.
The applications of virtual metrology and the concept of using virtual experiments (VEs) or digital twins in measurement tasks have gained significant interest in recent years. When real measurements are costly or scarce, a VE can replicate complex mathematical and physical models. As a digital representation of a measurement process, a VE can, among other use-cases, be used in an uncertainty evaluation. Here, simulated and real observations can be combined without explicit knowledge of a model for the measurand – a usual requirement for uncertainty evaluation according to the GUM and its supplement JCGM 101. Including the variability of repeated measurements in a VE can be a challenging task. However, metrologists are aware of the potential impact of repeatability conditions and usually have significant knowledge about the precision of their measurement instrument. In this work, we develop a Monte Carlo sampling approach and software to perform an uncertainty evaluation that includes such prior knowledge about the precision of the measurement device, requiring only evaluations of the VE. Using an example of a calibrated measurement application, we demonstrate that there is a notable benefit to using available prior knowledge during an uncertainty evaluation, particularly in the case of limited observations.
20 March 2025 in Measurement Science and Technology
In the data analysis of measurements, the simplified assumption of homoscedastic Gaussian noise is often made to account for the random fluctuations between observations. This may be an inadequate assumption which can deteriorate the results of a data analysis. Repeated measurements, which can be used to infer the true distribution of the data, may be inaccessible, making the true distribution hard to find. In such circumstances, thoroughly designed virtual experiments (VEs) can mimic real and possibly complex measurement processes to infer the true data distribution, which can subsequently be accounted for in an improved analysis of real observations. We explore the potential benefit of such an approach in terms of a metrological application, the tilted-wave interferometer (TWI). Our VE for the TWI yields not just the mean of the data, but also their physically modelled, random fluctuations arising in repeated observations. We use the virtual data to derive a statistical data model that includes correlations and heteroscedasticity. In applying a Bayesian data analysis procedure utilising said statistical model in conjunction with a vague prior for the quantity of interest, virtual data with a known ground truth are analysed and the quality of the resulting estimates are assessed. In addition, a comparison is carried out to the often-employed, simplified approach assuming homoscedastic, independent noise. We observe a significant improvement in the results when a more adequate statistical model for the data is utilised, along with a reliable uncertainty quantification. The work proposes the idea to extend the utilisation of a VE to inferring the noise characteristics of real observations, in turn leading to significantly improved data analysis procedures. The potential benefit is demonstrated to be substantial in terms of the considered metrological case study. Future research is discussed, including other ways that VEs could be used to further improve data analysis.
Virtual experiments (VEs), a modern tool in metrology, can be used to help perform an uncertainty evaluation for the measurand. Current guidelines in metrology do not cover the many possibilities to incorporate VEs into an uncertainty evaluation, and it is often difficult to assess if the intended use of a VE complies with said guidelines. In recent work, it was shown that a VE can be used in conjunction with real measurement data and a Monte Carlo procedure to produce equal results to a supplement of the Guide to the Expression of Uncertainty in Measurement. However, this was shown only for linear measurement models. In this work, we extend this Monte Carlo approach to a common class of non-linear measurement models and more complex VEs, providing a reference approach for suitable uncertainty evaluations involving VEs. Numerical examples are given to show that the theoretical derivations hold in a practical scenario.