IDEAS home Printed from https://ideas.repec.org/a/spr/jesaex/v7y2021i2d10.1007_s40881-021-00112-w.html
   My bibliography  Save this article

Running online experiments using web-conferencing software

Author

Listed:
  • Jiawei Li

    (The University of Michigan)

  • Stephen Leider

    (The University of Michigan)

  • Damian Beil

    (The University of Michigan)

  • Izak Duenyas

    (The University of Michigan)

Abstract

We report the results of a novel protocol for running online experiments using a combination of an online experimental platform in parallel with web-conferencing software in two formats—with and without subject webcams—to improve subjects’ attention and engagement. We compare the results between our online sessions with the offline (lab) sessions of the same experiment. We find that both online formats lead to comparable subject characteristics and performance as the offline (lab) experiment. However, the webcam-on protocol has less noisy data, and hence better statistical power, than the protocol without a webcam. The webcam-on protocol can detect reasonable effect sizes with a comparable sample size as in the offline (lab) protocol.

Suggested Citation

  • Jiawei Li & Stephen Leider & Damian Beil & Izak Duenyas, 2021. "Running online experiments using web-conferencing software," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 7(2), pages 167-183, December.
    • Handle: RePEc:spr:jesaex:v:7:y:2021:i:2:d:10.1007_s40881-021-00112-w
    DOI: 10.1007/s40881-021-00112-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40881-021-00112-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40881-021-00112-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John Horton & David Rand & Richard Zeckhauser, 2011. "The online laboratory: conducting experiments in a real labor market," Experimental Economics, Springer;Economic Science Association, vol. 14(3), pages 399-425, September.
    2. Ben Greiner, 2015. "Subject pool recruitment procedures: organizing experiments with ORSEE," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 1(1), pages 114-125, July.
    3. Antonio A. Arechar & Simon Gächter & Lucas Molleman, 2018. "Conducting interactive experiments online," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 99-131, March.
    4. Charles A. Holt & Susan K. Laury, 2002. "Risk Aversion and Incentive Effects," American Economic Review, American Economic Association, vol. 92(5), pages 1644-1655, December.
    5. Kovalchik, Stephanie & Camerer, Colin F. & Grether, David M. & Plott, Charles R. & Allman, John M., 2005. "Aging and decision making: a comparison between neurologically healthy elderly and young individuals," Journal of Economic Behavior & Organization, Elsevier, vol. 58(1), pages 79-94, September.
    6. Rachel Croson & Uri Gneezy, 2009. "Gender Differences in Preferences," Journal of Economic Literature, American Economic Association, vol. 47(2), pages 448-474, June.
    7. Carpenter, Jeffrey & Graham, Michael & Wolf, Jesse, 2013. "Cognitive ability and strategic sophistication," Games and Economic Behavior, Elsevier, vol. 80(C), pages 115-130.
    8. Gneezy, Uri & Rustichini, Aldo & Vostroknutov, Alexander, 2010. "Experience and insight in the Race game," Journal of Economic Behavior & Organization, Elsevier, vol. 75(2), pages 144-155, August.
    9. Duch, Matthias L. & Grossmann, Max R.P. & Lauer, Thomas, 2020. "z-Tree unleashed: A novel client-integrating architecture for conducting z-Tree experiments over the Internet," Journal of Behavioral and Experimental Finance, Elsevier, vol. 28(C).
    10. Shane Frederick, 2005. "Cognitive Reflection and Decision Making," Journal of Economic Perspectives, American Economic Association, vol. 19(4), pages 25-42, Fall.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Della Valle, Nives & D'Arcangelo, Chiara & Faillo, Marco, 2024. "Promoting pro-environmental choices while addressing energy poverty," Energy Policy, Elsevier, vol. 186(C).
    2. Werner, Tobias, 2021. "Algorithmic and human collusion," DICE Discussion Papers 372, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    3. Dmitri Bershadskyy & Sunil Ghadwal & Jannik Greif, 2024. "MTVE: Magdeburg tool for video experiments," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 10(2), pages 609-619, December.
    4. Jessica B. Hoel & Prachi Jain & Bridget Galaty, 2022. "JUST VENMO ME: Does form of payment affect risk taking and intertemporal choice?," Journal of the Economic Science Association, Springer;Economic Science Association, vol. 8(1), pages 16-33, December.
    5. Abraham, Diya & Corazzini, Luca & Fišar, Miloš & Reggiani, Tommaso, 2023. "Coordinating donations via an intermediary: The destructive effect of a sunk overhead cost," Journal of Economic Behavior & Organization, Elsevier, vol. 211(C), pages 287-304.
    6. Guo, Yiting & Shachat, Jason & Walker, Matthew J. & Wei, Lijia, 2023. "On the generalizability of using mobile devices to conduct economic experiments," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 106(C).
    7. Masiliūnas, Aidas, 2023. "Learning in rent-seeking contests with payoff risk and foregone payoff information," Games and Economic Behavior, Elsevier, vol. 140(C), pages 50-72.
    8. Pin, Paolo & Rotesi, Tiziano, 2023. "App-based experiments," Journal of Economic Psychology, Elsevier, vol. 99(C).
    9. Daniel Borup & Jorge Wolfgang Hansen & Benjamin Dybro Liengaard & Erik Christian Montes Schütte, 2023. "Quantifying investor narratives and their role during COVID‐19," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(4), pages 512-532, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Prissé, Benjamin & Jorrat, Diego, 2022. "Lab vs online experiments: No differences," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 100(C).
    2. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    3. Carlos Alós-Ferrer & Johannes Buckenmaier, 2021. "Cognitive sophistication and deliberation times," Experimental Economics, Springer;Economic Science Association, vol. 24(2), pages 558-592, June.
    4. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2022. "Comparing behavior between a large sample of smart students and a representative sample of Japanese adults," ISER Discussion Paper 1160, Institute of Social and Economic Research, Osaka University.
    5. Choo, Lawrence & Kaplan, Todd R. & Zhou, Xiaoyu, 2019. "Can auctions select people by their level-k types?," MPRA Paper 95987, University Library of Munich, Germany.
    6. Ciril Bosch-Rosa & Thomas Meissner & Antoni Bosch-Domènech, 2018. "Cognitive bubbles," Experimental Economics, Springer;Economic Science Association, vol. 21(1), pages 132-153, March.
    7. Nobuyuki Hanaki & Keigo Inukai & Takehito Masuda & Yuta Shimodaira, 2021. "Participants’ Characteristics at ISER-Lab in 2020," ISER Discussion Paper 1141, Institute of Social and Economic Research, Osaka University.
    8. Lucks, Konstantin, 2016. "The Impact of Self-Control on Investment Decisions," MPRA Paper 73099, University Library of Munich, Germany.
    9. Taylor, Matthew P., 2020. "Heterogeneous motivation and cognitive ability in the lab," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 85(C).
    10. Prissé, Benjamin & Jorrat, Diego, 2021. "Lack of Control: An experiment," MPRA Paper 109918, University Library of Munich, Germany.
    11. Amador, Luis & Brañas-Garza, Pablo & Espín, Antonio M. & Garcia, Teresa & Hernández, Ana, 2019. "Consistent and inconsistent choices under uncertainty: The role of cognitive abilities," MPRA Paper 95178, University Library of Munich, Germany.
    12. Zylbersztejn, Adam & Babutsidze, Zakaria & Hanaki, Nobuyuki & Hopfensitz, Astrid, 2024. "How beautiful people see the world: Cooperativeness judgments of and by beautiful people," Journal of Economic Behavior & Organization, Elsevier, vol. 218(C), pages 296-308.
    13. Lorko, Matej & Servátka, Maroš & Zhang, Le, 2023. "Hidden inefficiency: Strategic inflation of project schedules," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 313-326.
    14. Haucap, Justus & Heldman, Christina & Rau, Holger A., 2024. "Gender and cooperation in the presence of negative externalities," Games and Economic Behavior, Elsevier, vol. 147(C), pages 148-169.
    15. Helena Fornwagner & Monika Pompeo & Nina Serdarevic, 2023. "Choosing Competition on Behalf of Someone Else," Management Science, INFORMS, vol. 69(3), pages 1555-1574, March.
    16. Cason, Timothy N. & Mui, Vai-Lam, 2019. "Individual versus group choices of repeated game strategies: A strategy method approach," Games and Economic Behavior, Elsevier, vol. 114(C), pages 128-145.
    17. Lauber, Arne & March, Christoph & Sahm, Marco, 2023. "Optimal and fair prizing in sequential round-robin tournaments: Experimental evidence," Games and Economic Behavior, Elsevier, vol. 141(C), pages 30-51.
    18. Helena Fornwagner & Monika Pompeo & Nina Serdarevic, 2020. "Him or her? Choosing competition on behalf of someone else," Discussion Papers 2020-13, The Centre for Decision Research and Experimental Economics, School of Economics, University of Nottingham.
    19. Grandjean, Gilles & Lefebvre, Mathieu & Mantovani, Marco, 2022. "Preferences and strategic behavior in public goods games," Journal of Economic Behavior & Organization, Elsevier, vol. 195(C), pages 171-185.
    20. Tamás Csermely & Alexander Rabas, 2016. "How to reveal people’s preferences: Comparing time consistency and predictive power of multiple price list risk elicitation methods," Journal of Risk and Uncertainty, Springer, vol. 53(2), pages 107-136, December.

    More about this item

    Keywords

    Online experiment; Experimental methodology; ZTREE unleashed; Web-conferencing software; Webcam; Selection bias; Subject performance;
    All these keywords.

    JEL classification:

    • C90 - Mathematical and Quantitative Methods - - Design of Experiments - - - General
    • C91 - Mathematical and Quantitative Methods - - Design of Experiments - - - Laboratory, Individual Behavior
    • D91 - Microeconomics - - Micro-Based Behavioral Economics - - - Role and Effects of Psychological, Emotional, Social, and Cognitive Factors on Decision Making

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jesaex:v:7:y:2021:i:2:d:10.1007_s40881-021-00112-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.