On the Generalizability of Using Mobile Devices to Conduct Economic Experiments

Recent technological advances enable the implementation of online, field and hybrid experiments using mobile devices. Mobile devices enable sampling of incentivized decisions in more representative samples, consequently increasing the generalizability of results. Generalizability might be compromised, however, if the device is a relevant behavioural confound. This paper reports on a battery of common economic games and decision-making tasks in which we systematically randomize the decision-making device (computer versus mobile phone) and the laboratory setup (physical versus online). The results offer broad support for conducting decision experiments using mobile devices. For six out of eight tasks, we find robust null results in terms of average treatment effects and variability. This should give researchers confidence to conduct studies out-of-laboratory via mobile phones. However, we find two caveats. First, with respect to decisions, subjects using a mobile phone are significantly more risk averse and offer less during bargaining. Second, decision response times and the time taken to read instructions are significantly shorter for the online-mobile treatment. These caveats suggest the importance of ensuring device consistency across treatments in the digital age of experimentation.