Inference Based on the Best-Fitting Model Can Contribute to the Replication Crisis: Assessing Model Selection Uncertainty Using a Bootstrap Approach

TitleInference Based on the Best-Fitting Model Can Contribute to the Replication Crisis: Assessing Model Selection Uncertainty Using a Bootstrap Approach
Publication TypeJournal Article
Year of Publication2016
AuthorsLubke, G, Campbell, I
JournalStructural Equation Modeling: A Multidisciplinary Journal
Volume23
Pagination479-490
Abstract

Inferences and conclusions drawn from model fitting analyses are commonly based on a single “best fitting” model. If model selection and inference are carried out using the same data model selection, uncertainty is ignored. We illustrate the Type I error inflation that can result from using the same data for model selection and inference, and we then propose a simple bootstrap-based approach to quantify model selection uncertainty in terms of model selection rates. A selection rate can be interpreted as an estimate of the replication probability of a fitted model. The benefits of bootstrapping model selection uncertainty are demonstrated in growth mixture analyses of data from the National Longitudinal Study of Youth, and a 2-group measurement invariance analysis of the Holzinger–Swineford data.

URLhttps://doi.org/10.1080/10705511.2016.1141355
DOI10.1080/10705511.2016.1141355