Behavioral changes in different designs of search experiments

While search experiments are available in several designs, growing experimental evidence suggests that individual search behavior depends on design details. We conduct an experiment providing the first categorization and comparison of several search experiment designs widely accepted in search studies. These designs can be categorized as passive, quasi-active, and active, according to the degree of flexibility in decision-making regarding the search. Despite the experiment being based on an identical model, we found significant differences at the aggregate- and individual-level in the results across designs. The average number of searches was the highest and closest to the theoretical value in the active design. Compared with the active design, subjects searched significantly less in the quasiactive and passive designs. The results indicate that the widely accepted design, wherein subjects make decisions based on a given offer rather than choosing among potential alternatives themselves, may have unexpected effects on subjects’ behavior. Furthermore, subjects’ risk aversion has a significant effect only in the passive design, suggesting that out-of-model factors specific to that design may influence behavior through risk preferences. Other methodological implications for search experiments are also provided.