When Does Real Become Consequential in Non-hypothetical Choice Experiments?

The proneness of stated preference methods to hypothetical bias has increased the popularity of incentivized studies, in particular the use of real choice experiments (RCE). Challenges of RCE include the lack of engagement with the choice task by some subjects, and that some of the product alternatives may not be available in order to incentivize all the choices. This issue brings to question whether the proportion of available products influences the results of the RCE. Would the subjects' engagement change? Using an induced value choice experiment with a profit maximization optimal strategy for agents, we varied the number of potentially binding alternatives in four treatments. Our results suggest that incentives matter, as the percentage of optimal choices was lowest in the hypothetical treatment. Interestingly, however, we do not find statistically significant differences in the number of optimal choices between the incentivized treatments, regardless of the number of potentially binding alternatives used in our treatments. This suggests that practitioners could conduct incentivized RCE without the need to have all the product alternatives be made available in the study. Furthermore, we explore the interaction of incentives with subjects' numerical ability and individual reflective state. Both are also shown to influence how incentives impact performance, shedding some light on what individual characteristics to look for when conducting valuation research.