IDEAS home Printed from https://ideas.repec.org/a/spr/jesaex/v7y2021i2d10.1007_s40881-021-00112-w.html
   My bibliography  Save this article

Running online experiments using web-conferencing software

Author

Listed:
  • Jiawei Li

    (The University of Michigan)

  • Stephen Leider

    (The University of Michigan)

  • Damian Beil

    (The University of Michigan)

  • Izak Duenyas

    (The University of Michigan)

Abstract

We report the results of a novel protocol for running online experiments using a combination of an online experimental platform in parallel with web-conferencing software in two formats—with and without subject webcams—to improve subjects’ attention and engagement. We compare the results between our online sessions with the offline (lab) sessions of the same experiment. We find that both online formats lead to comparable subject characteristics and performance as the offline (lab) experiment. However, the webcam-on protocol has less noisy data, and hence better statistical power, than the protocol without a webcam. The webcam-on protocol can detect reasonable effect sizes with a comparable sample size as in the offline (lab) protocol.

Suggested Citation

  • Jiawei Li & Stephen Leider & Damian Beil & Izak Duenyas, 2021. "Running online experiments using web-conferencing software," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 7(2), pages 167-183, December.
    • Handle: RePEc:spr:jesaex:v:7:y:2021:i:2:d:10.1007_s40881-021-00112-w
    DOI: 10.1007/s40881-021-00112-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40881-021-00112-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40881-021-00112-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    2. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    3. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    4. Carpenter, Jeffrey & Graham, Michael & Wolf, Jesse, 2013. "Cognitive ability and strategic sophistication," Games and Economic Behavior, Elsevier, vol. 80(C), pages 115-130.
    5. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    6. Gneezy, Uri & Rustichini, Aldo & Vostroknutov, Alexander, 2010. "Experience and insight in the Race game," Journal of Economic Behavior & Organization, Elsevier, vol. 75(2), pages 144-155, August.
    7. Duch, Matthias L. & Grossmann, Max R.P. & Lauer, Thomas, 2020. "z-Tree unleashed: A novel client-integrating architecture for conducting z-Tree experiments over the Internet," Journal of Behavioral and Experimental Finance, Elsevier, vol. 28(C).
    8. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    9. Kovalchik, Stephanie & Camerer, Colin F. & Grether, David M. & Plott, Charles R. & Allman, John M., 2005. "Aging and decision making: a comparison between neurologically healthy elderly and young individuals," Journal of Economic Behavior & Organization, Elsevier, vol. 58(1), pages 79-94, September.
    10. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Della Valle, Nives & D'Arcangelo, Chiara & Faillo, Marco, 2024. "Promoting pro-environmental choices while addressing energy poverty," Energy Policy, Elsevier, vol. 186(C).
    2. Werner, Tobias, 2021. "Algorithmic and human collusion," DICE Discussion Papers 372, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    3. Jessica B. Hoel & Prachi Jain & Bridget Galaty, 2022. "JUST VENMO ME: Does form of payment affect risk taking and intertemporal choice?," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 8(1), pages 16-33, December.
    4. Abraham, Diya & Corazzini, Luca & Fišar, Miloš & Reggiani, Tommaso, 2023. "Coordinating donations via an intermediary: The destructive effect of a sunk overhead cost," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 287-304.
    5. Guo, Yiting & Shachat, Jason & Walker, Matthew J. & Wei, Lijia, 2023. "On the generalizability of using mobile devices to conduct economic experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    6. Masiliūnas, Aidas, 2023. "Learning in rent-seeking contests with payoff risk and foregone payoff information," Games and Economic Behavior, Elsevier, vol. 140(C), pages 50-72.
    7. Pin, Paolo & Rotesi, Tiziano, 2023. "App-based experiments," Journal of Economic Psychology, Elsevier, vol. 99(C).
    8. Daniel Borup & Jorge Wolfgang Hansen & Benjamin Dybro Liengaard & Erik Christian Montes Schütte, 2023. "Quantifying investor narratives and their role during COVID‐19," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 512-532, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ciril Bosch-Rosa & Thomas Meissner & Antoni Bosch-Domènech, 2018. "Cognitive bubbles," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 132-153, March.
    2. Prissé, Benjamin & Jorrat, Diego, 2022. "Lab vs online experiments: No differences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    3. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    4. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2021. "Participants’ Characteristics at ISER-Lab in 2020," ISER Discussion Paper 1141, Institute of Social and Economic Research, Osaka University.
    5. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    6. Lucks, Konstantin, 2016. "The Impact of Self-Control on Investment Decisions," MPRA Paper 73099, University Library of Munich, Germany.
    7. Taylor, Matthew P., 2020. "Heterogeneous motivation and cognitive ability in the lab," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    8. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2022. "Comparing behavior between a large sample of smart students and a representative sample of Japanese adults," ISER Discussion Paper 1160, Institute of Social and Economic Research, Osaka University.
    9. Prissé, Benjamin & Jorrat, Diego, 2021. "Lack of Control: An experiment," MPRA Paper 109918, University Library of Munich, Germany.
    10. Amador, Luis & Brañas-Garza, Pablo & Espín, Antonio M. & Garcia, Teresa & Hernández, Ana, 2019. "Consistent and inconsistent choices under uncertainty: The role of cognitive abilities," MPRA Paper 95178, University Library of Munich, Germany.
    11. Choo, Lawrence & Kaplan, Todd R. & Zhou, Xiaoyu, 2019. "Can auctions select people by their level-k types?," MPRA Paper 95987, University Library of Munich, Germany.
    12. Zylbersztejn, Adam & Babutsidze, Zakaria & Hanaki, Nobuyuki & Hopfensitz, Astrid, 2024. "How beautiful people see the world: Cooperativeness judgments of and by beautiful people," Journal of Economic Behavior & Organization, Elsevier, vol. 218(C), pages 296-308.
    13. Lorko, Matej & Servátka, Maroš & Zhang, Le, 2023. "Hidden inefficiency: Strategic inflation of project schedules," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 313-326.
    14. Nobuyuki Hanaki & Eizo Akiyama & Yukihiko Funaki & Ryuichiro Ishikawa, 2017. "Diversity in Cognitive Ability Enlarges Mispricing in Experimental Asset Markets," GREDEG Working Papers 2017-08, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    15. Lohse, Johannes, 2016. "Smart or selfish – When smart guys finish nice," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 64(C), pages 28-40.
    16. Haucap, Justus & Heldman, Christina & Rau, Holger A., 2024. "Gender and cooperation in the presence of negative externalities," Games and Economic Behavior, Elsevier, vol. 147(C), pages 148-169.
    17. Helena Fornwagner & Monika Pompeo & Nina Serdarevic, 2023. "Choosing Competition on Behalf of Someone Else," Management Science, INFORMS, vol. 69(3), pages 1555-1574, March.
    18. Cason, Timothy N. & Mui, Vai-Lam, 2019. "Individual versus group choices of repeated game strategies: A strategy method approach," Games and Economic Behavior, Elsevier, vol. 114(C), pages 128-145.
    19. Lauber, Arne & March, Christoph & Sahm, Marco, 2023. "Optimal and fair prizing in sequential round-robin tournaments: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 141(C), pages 30-51.
    20. Ahrens, Steffen & Bitter, Lea & Bosch-Rosa, Ciril, 2023. "Coordination under loss contracts," Games and Economic Behavior, Elsevier, vol. 137(C), pages 270-293.

    More about this item

    Keywords

    Online experiment; Experimental methodology; ZTREE unleashed; Web-conferencing software; Webcam; Selection bias; Subject performance;
    All these keywords.

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jesaex:v:7:y:2021:i:2:d:10.1007_s40881-021-00112-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.