Case Study in Combining Physical and Computer Experiments

Abstract

Estimation of computer model parameters using field data is sometimes attempted while simultaneously allowing for model bias. One paper reports that simultaneous estimation of a bias vector and a scalar calibration parameter, which results in a "calibrated computer model," can be sensitive to assumptions made prior to data collection. Other papers show that "calibrated computer models" can lead to improved response prediction, as measured by the root mean squared prediction error (RMSE). This paper uses a simulated case study to show that the RMSE from a purely empirical prediction option (local kernel smoothing) can be smaller than the RMSE from a "calibrated computer model" option. Therefore, although we endorse "calibrated computer models," we point out that purely empirical models can provide competitive predictions in some cases.