IDEAS home Printed from https://ideas.repec.org/a/eee/beexfi/v23y2019icp138-160.html
   My bibliography  Save this article

Web-based experimental economics software: How do they compare to desirable features?

Author

Listed:
  • Chan, Shu Wing
  • Schilizzi, Steven
  • Iftekhar, Md Sayed
  • Da Silva Rosa, Raymond

Abstract

Web-based experiments that cut across the lab vs. field distinction are increasingly popular with economists. However, non-standardized software features and services hinder comparability and replication. This study reviews a wide selection of experimental economics software packages and evaluates them against criteria based on the logistics and operational requirements of economic experiments. We find that oTree and SoPHIE rank highest across criteria, but Veconlab and classEx might be suitable for those with a dominant need for a large library of ready-made experiments. We find a portability gap: no presently available software allows portability of experiments across platforms because of technical complexity and the challenging coordination needs of experimental economists. As a result, experiments may be replicated only on the same platform or with the same software, but general replicability is slow and costly. This constrains the development of experimental economics as a replicable science.

Suggested Citation

  • Chan, Shu Wing & Schilizzi, Steven & Iftekhar, Md Sayed & Da Silva Rosa, Raymond, 2019. "Web-based experimental economics software: How do they compare to desirable features?," Journal of Behavioral and Experimental Finance, Elsevier, vol. 23(C), pages 138-160.
    • Handle: RePEc:eee:beexfi:v:23:y:2019:i:c:p:138-160
    DOI: 10.1016/j.jbef.2019.04.007
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S221463501830090X
    Download Restriction: no
    File URL: https://libkey.io/10.1016/j.jbef.2019.04.007?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    2. Paolacci, Gabriele & Chandler, Jesse & Ipeirotis, Panagiotis G., 2010. "Running experiments on Amazon Mechanical Turk," Judgment and Decision Making, Cambridge University Press, vol. 5(5), pages 411-419, August.
    3. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    4. Paul Gertler & Sebastian Galiani & Mauricio Romero, 2018. "How to make replication the norm," Nature, Nature, vol. 554(7693), pages 417-419, February.
    5. Bock, Olaf & Baetge, Ingmar & Nicklisch, Andreas, 2014. "hroot: Hamburg Registration and Organization Online Tool," European Economic Review, Elsevier, vol. 71(C), pages 117-120.
    6. Giamattei, Marcus & Lambsdorff, Johann Graf, 2019. "classEx — an online tool for lab-in-the-field experiments with smartphones," Journal of Behavioral and Experimental Finance, Elsevier, vol. 22(C), pages 223-231.
    7. repec:cup:judgdm:v:5:y:2010:i:5:p:411-419 is not listed on IDEAS
    8. Seithe, Mirko, 2012. "Introducing the Bonn Experiment System (BoXS)," Bonn Econ Discussion Papers 01/2012, University of Bonn, Bonn Graduate School of Economics (BGSE).
    9. Chen, Daniel L. & Schonger, Martin & Wickens, Chris, 2016. "oTree—An open-source platform for laboratory, online, and field experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 9(C), pages 88-97.
    10. Camerer, Colin & Dreber, Anna & Forsell, Eskil & Ho, Teck-Hua & Huber, Jurgen & Johannesson, Magnus & Kirchler, Michael & Almenberg, Johan & Altmejd, Adam & Chan, Taizan & Heikensten, Emma & Holzmeist, 2016. "Evaluating replicability of laboratory experiments in Economics," MPRA Paper 75461, University Library of Munich, Germany.
    11. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kumar, Satish & Rao, Sandeep & Goyal, Kirti & Goyal, Nisha, 2022. "Journal of Behavioral and Experimental Finance: A bibliometric overview," Journal of Behavioral and Experimental Finance, Elsevier, vol. 34(C).
    2. Marcus Giamattei & Kyanoush Seyed Yahosseini & Simon Gächter & Lucas Molleman, 2020. "LIONESS Lab: a free web-based platform for conducting interactive experiments online," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 95-111, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcus Giamattei & Kyanoush Seyed Yahosseini & Simon Gächter & Lucas Molleman, 2020. "LIONESS Lab: a free web-based platform for conducting interactive experiments online," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 6(1), pages 95-111, June.
    2. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    3. Molnar, Andras, 2019. "SMARTRIQS: A Simple Method Allowing Real-Time Respondent Interaction in Qualtrics Surveys," Journal of Behavioral and Experimental Finance, Elsevier, vol. 22(C), pages 161-169.
    4. Gächter, Simon & Kölle, Felix & Quercia, Simone, 2022. "Preferences and perceptions in Provision and Maintenance public goods," Games and Economic Behavior, Elsevier, vol. 135(C), pages 338-355.
    5. Florian Baumann & Tim Friehe & Pascal Langenbach, 2020. "Fines versus Damages: Experimental Evidence on Care Investments," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2020_08, Max Planck Institute for Research on Collective Goods, revised Mar 2024.
    6. Lorko, Matej & Servátka, Maroš & Zhang, Le, 2023. "Hidden inefficiency: Strategic inflation of project schedules," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 313-326.
    7. Boukouras, Aristotelis & Jennings, Will & Li, Lunzheng & Maniadis, Zacharias, 2023. "Can biased polls distort electoral results? Evidence from the lab," European Journal of Political Economy, Elsevier, vol. 78(C).
    8. Dato, Simon & Feess, Eberhard & Nieken, Petra, 2019. "Lying and reciprocity," Games and Economic Behavior, Elsevier, vol. 118(C), pages 193-218.
    9. Danková, Katarína & Servátka, Maroš, 2019. "Gender robustness of overconfidence and excess entry," Journal of Economic Psychology, Elsevier, vol. 72(C), pages 179-199.
    10. Gächter, Simon & Lee, Kyeongtae & Sefton, Martin & Weber, Till O., 2024. "The role of payoff parameters for cooperation in the one-shot Prisoner's Dilemma," European Economic Review, Elsevier, vol. 166(C).
    11. König-Kersting, Christian & Trautmann, Stefan T. & Vlahu, Razvan, 2022. "Bank instability: Interbank linkages and the role of disclosure," Journal of Banking & Finance, Elsevier, vol. 134(C).
    12. Fischbacher, Urs & Hausfeld, Jan & Renerte, Baiba, 2022. "Strategic incentives undermine gaze as a signal of prosocial motives," Games and Economic Behavior, Elsevier, vol. 136(C), pages 63-91.
    13. Schmidt, Robert J., 2019. "Do injunctive or descriptive social norms elicited using coordination games better explain social preferences?," Working Papers 0668, University of Heidelberg, Department of Economics.
    14. Fehr, Dietmar & Rau, Hannes & Trautmann, Stefan T. & Xu, Yilong, 2020. "Inequality, fairness and social capital," European Economic Review, Elsevier, vol. 129(C).
    15. Przepiorka, Wojtek, 2023. "Laboratory experiments," SocArXiv 9cxq2, Center for Open Science.
    16. Hippel, Svenja & Hoeppner, Sven, 2019. "Biased judgements of fairness in bargaining: A replication in the laboratory," International Review of Law and Economics, Elsevier, vol. 58(C), pages 63-74.
    17. Florian Engl & Arno Riedl & Roberto Weber, 2021. "Spillover Effects of Institutions on Cooperative Behavior, Preferences, and Beliefs," American Economic Journal: Microeconomics, American Economic Association, vol. 13(4), pages 261-299, November.
    18. Heinicke, Franziska & König-Kersting, Christian & Schmidt, Robert, 2022. "Injunctive vs. descriptive social norms and reference group dependence," Journal of Economic Behavior & Organization, Elsevier, vol. 195(C), pages 199-218.
    19. Utz Weitzel & Christoph Huber & Jürgen Huber & Michael Kirchler & Florian Lindner & Julia Rose & Lauren Cohen, 2020. "Bubbles and Financial Professionals," The Review of Financial Studies, Society for Financial Studies, vol. 33(6), pages 2659-2696.
    20. Dohmen, Thomas & Quercia, Simone & Willrodt, Jana, 2023. "A note on salience of own preferences and the consensus effect," Journal of Economic Behavior & Organization, Elsevier, vol. 209(C), pages 15-21.

    More about this item

    Keywords

    Experimental economics; Web-based; Software; Online experiments; Web-based experiments; Economic experiments;
    All these keywords.

    JEL classification:

    • C81 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Methodology for Collecting, Estimating, and Organizing Microeconomic Data; Data Access
    • C88 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Other Computer Software

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:beexfi:v:23:y:2019:i:c:p:138-160. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/journal-of-behavioral-and-experimental-finance .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.