Characterisation and Calibration of Multiversal Models

The Multiverse is a framework to design multi-model estimation where data are fit on many connected specifications of the same abstract model instead of a singular or a small selection of specifications. Differently from canonical multi-model, in Multiverse the probabilities of the specifications to be included in the analysis are never assumed independent of each other. Grounded in this consideration, this study provides a compact statistical characterisation of the process of elicitation of the specifications in Multiverse Analysis and conceptually adjacent methods, connecting previous insights from meta-analytical Statistics, model averaging, Network Theory, Information Theory, and Causal Inference. The topic of calibration of the multiversal estimates is treated with references to the adoption Bayesian Model Averaging vs. alternatives. In an application, it is checked the theory that Bayesian Model Averaging reduces error for well-specified multiversal models, but it amplifies it when a collider variable is included in the multiversal model. In well-specified models, alternatives do not perform significantly better than Uniform weighting of the estimates, so the adoption of a gold standard remains ambiguous. Normative implications for misinterpretation of the epistemic value of Multiverse Analysis and the connection of the proposed characterisation and future directions of research are discussed.