Abstract

Model misspecification analysis strategies, such as anomaly detection, modelvalidation, and model comparison are a key component of scientific modeldevelopment. Over the last few years, there has been a rapid rise in the use ofsimulation-based inference (SBI) techniques for Bayesian parameter estimation,applied to increasingly complex forward models. To move towards fullysimulation-based analysis pipelines, however, there is an urgent need for acomprehensive simulation-based framework for model misspecification analysis.In this work, we provide a solid and flexible foundation for a wide range ofmodel discrepancy analysis tasks, using distortion-driven modelmisspecification tests. From a theoretical perspective, we introduce thestatistical framework built around performing many hypothesis tests fordistortions of the simulation model. We also make explicit analytic connectionsto classical techniques: anomaly detection, model validation, andgoodness-of-fit residual analysis. Furthermore, we introduce an efficientself-calibrating training algorithm that is useful for practitioners. Wedemonstrate the performance of the framework in multiple scenarios, making theconnection to classical results where they are valid. Finally, we show how toconduct such a distortion-driven model misspecification test for realgravitational wave data, specifically on the event GW150914.