Subjects, Trials, and Levels: Statistical Power in Conjoint Experiments

Despite the importance of power analysis for survey-experimental techniques, power considerations are often disregraded in the design of conjoint experiments. The main goal of this article is to provide rigorous guidance on how the number of experimental conditions, trials, and subjects impact the statistical power of conjoint experiments. To this end, we first conducted an extensive literature review to gauge the experimental designs typically employed in conjoint studies and the plausible effect sizes discovered in the literature. Using the information gather from the literature review, we explore the statistical properties of a wide range of commonly employed designs using a simulation-based framework that employs a flexible data generating model. Results show that—even with relatively large sample size and the number of trials—conjoint experiments are not well suited to draw inferences for designs with large numbers of experimental conditions (> 15) and relatively small effect sizes (< 1000). Based on our simulation results, we develop a web application that can be used by researches to perform a priori power analysis and hence achieve adequate design for future conjoint experiments.