IDEAS home Printed from https://ideas.repec.org/p/hal/wpaper/halshs-03036837.html
   My bibliography  Save this paper

The Use of Experimental Methods by IS Scholars: An Illustrated Typology

Author

Listed:
  • Marta Ballatore

    (GREDEG - Groupe de Recherche en Droit, Economie et Gestion - UNS - Université Nice Sophia Antipolis (1965 - 2019) - CNRS - Centre National de la Recherche Scientifique - UniCA - Université Côte d'Azur)

  • Agnès Festré

    (GREDEG - Groupe de Recherche en Droit, Economie et Gestion - UNS - Université Nice Sophia Antipolis (1965 - 2019) - CNRS - Centre National de la Recherche Scientifique - UniCA - Université Côte d'Azur)

  • Lise Arena

    (GREDEG - Groupe de Recherche en Droit, Economie et Gestion - UNS - Université Nice Sophia Antipolis (1965 - 2019) - CNRS - Centre National de la Recherche Scientifique - UniCA - Université Côte d'Azur)

Abstract

This article aims at making an updated typology of recent experimental studies in the IS literature on the period 1999-2019. Based on a full-text search within the Association for Information Systems (AIS) "basket" of eight top IS journals (EJIS, ISR, JAIS, ISJ, JIT, JMIS, JSIS and MISQ), this research gathered 392 articles and highlights the use of 5 different types of experiments in IS, mainly: artificial simulations, laboratory experiments, field experiments, online experiments (scenario simulation game-based; brainstorming-based. . . ) and natural experiments. Each category is discussed through the perspective of its degree of control, and technological realism. Results show the significant predominance of laboratory experiments over field and natural experiments on the period. This, in turn, stresses the preferred tendency followed by IS scholars to perceive experimental methods as a way to control the source of variations of variables under study. In addition, this paper provides a better understanding of the context of use of a specific experimental method. Overall, it is shown that laboratory experiments (including scenario-based lab experiments) are mainly used, in a deterministic manner, to assess or test the impact of an IS on human decision-making or behaviour. By contrast, artificial simulations experiments are more appropriate to study emergent phenomena and to make predictions, often providing key insights about quality and effectiveness of IS.

Suggested Citation

  • Marta Ballatore & Agnès Festré & Lise Arena, 2020. "The Use of Experimental Methods by IS Scholars: An Illustrated Typology," Working Papers halshs-03036837, HAL.
  • Handle: RePEc:hal:wpaper:halshs-03036837
    Note: View the original document on HAL open archive server: https://shs.hal.science/halshs-03036837
    as

    Download full text from publisher

    File URL: https://shs.hal.science/halshs-03036837/document
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kar Yan Tam & Shuk Ying Ho, 2005. "Web Personalization as a Persuasion Strategy: An Elaboration Likelihood Model Perspective," Information Systems Research, INFORMS, vol. 16(3), pages 271-291, September.
    2. Anuj Kumar & Kartik Hosanagar, 2019. "Measuring the Value of Recommendation Links on Product Demand," Information Systems Research, INFORMS, vol. 30(3), pages 819-838, September.
    3. Amy Wenxuan Ding & Shibo Li & Patrali Chatterjee, 2015. "Learning User Real-Time Intent for Optimal Dynamic Web Page Transformation," Information Systems Research, INFORMS, vol. 26(2), pages 339-359, June.
    4. Angelika Dimoka & Paul A. Pavlou & Fred D. Davis, 2011. "Research Commentary ---NeuroIS: The Potential of Cognitive Neuroscience for Information Systems Research," Information Systems Research, INFORMS, vol. 22(4), pages 687-702, December.
    5. Weiquan Wang & May Wang, 2019. "Effects of Sponsorship Disclosure on Perceived Integrity of Biased Recommendation Agents: Psychological Contract Violation and Knowledge-Based Trust Perspectives," Information Systems Research, INFORMS, vol. 30(2), pages 507-522, June.
    6. Ray M. Chang & Wonseok Oh & Alain Pinsonneault & Dowan Kwon, 2010. "A Network Perspective of Digital Competition in Online Advertising Industries: A Simulation-Based Approach," Information Systems Research, INFORMS, vol. 21(3), pages 571-593, September.
    7. Matthew J. Hashim & Karthik N. Kannan & Sandra Maximiano, 2017. "Information Feedback, Targeting, and Coordination: An Experimental Study," Information Systems Research, INFORMS, vol. 28(2), pages 289-308, June.
    8. Vernon L. Smith, 1994. "Economics in the Laboratory," Journal of Economic Perspectives, American Economic Association, vol. 8(1), pages 113-131, Winter.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bo Zhou & Tianxin Zou, 2023. "Competing for Recommendations: The Strategic Impact of Personalized Product Recommendations in Online Marketplaces," Marketing Science, INFORMS, vol. 42(2), pages 360-376, March.
    2. Xueming Luo & Xianghua Lu & Jing Li, 2019. "When and How to Leverage E-commerce Cart Targeting: The Relative and Moderated Effects of Scarcity and Price Incentives with a Two-Stage Field Experiment and Causal Forest Optimization," Information Systems Research, INFORMS, vol. 30(4), pages 1203-1227, December.
    3. Randall Holcombe, 2009. "The behavioral foundations of Austrian economics," The Review of Austrian Economics, Springer;Society for the Development of Austrian Economics, vol. 22(4), pages 301-313, December.
    4. Bateman, Ian J. & Langford, Ian H. & Jones, Andrew P. & Kerr, Geoffrey N., 2001. "Bound and path effects in double and triple bounded dichotomous choice contingent valuation," Resource and Energy Economics, Elsevier, vol. 23(3), pages 191-213, July.
    5. Gabriele Camera & Cary Deck & David Porter, 2020. "Do economic inequalities affect long-run cooperation and prosperity?," Experimental Economics, Springer;Economic Science Association, vol. 23(1), pages 53-83, March.
    6. C.Mónica Capra & Tomomi Tanaka & ColinF. Camerer & Lauren Feiler & Veronica Sovero & CharlesN. Noussair, 2009. "The Impact of Simple Institutions in Experimental Economies with Poverty Traps," Economic Journal, Royal Economic Society, vol. 119(539), pages 977-1009, July.
    7. Frances X. Frei & Patrick T. Harker & Larry W. Hunter, 1995. "Performance in Consumer Financial Services Organizations: Framework and Results from the Pilot Study," Center for Financial Institutions Working Papers 95-03, Wharton School Center for Financial Institutions, University of Pennsylvania.
    8. Francis Green & Nicholas Tsitsianis, 2004. "Can the Changing Nature of Jobs Account for National Trends in Job Satisfaction?," Studies in Economics 0406, School of Economics, University of Kent.
    9. Amy Wenxuan Ding & Shibo Li & Patrali Chatterjee, 2015. "Learning User Real-Time Intent for Optimal Dynamic Web Page Transformation," Information Systems Research, INFORMS, vol. 26(2), pages 339-359, June.
    10. Mahesh Balan U & Saji K. Mathew, 2021. "Personalize, Summarize or Let them Read? A Study on Online Word of Mouth Strategies and Consumer Decision Process," Information Systems Frontiers, Springer, vol. 23(3), pages 627-647, June.
    11. Nitin Walia & Mark Srite & Wendy Huddleston, 2016. "Eyeing the web interface: the influence of price, product, and personal involvement," Electronic Commerce Research, Springer, vol. 16(3), pages 297-333, September.
    12. Jason Aimone & Daniel Houser, 2012. "What you don’t know won’t hurt you: a laboratory analysis of betrayal aversion," Experimental Economics, Springer;Economic Science Association, vol. 15(4), pages 571-588, December.
    13. Deck, Cary & Murphy, James J., 2019. "Donors change both their level and pattern of giving in response to contests among charities," European Economic Review, Elsevier, vol. 112(C), pages 91-106.
    14. Hans‐Theo Normann & Roberto Ricciuti, 2009. "Laboratory Experiments For Economic Policy Making," Journal of Economic Surveys, Wiley Blackwell, vol. 23(3), pages 407-432, July.
    15. Pluyter, J.R., 2012. "Designing immersive surgical training against information technology-related overload in the operating room," Other publications TiSEM d48c5727-92fd-41b1-be5d-5, Tilburg University, School of Economics and Management.
    16. Alessandra Cassar & Giovanna d'Adda & Pauline Grosjean, 2014. "Institutional Quality, Culture, and Norms of Cooperation: Evidence from Behavioral Field Experiments," Journal of Law and Economics, University of Chicago Press, vol. 57(3), pages 821-863.
    17. Khim-Yong Goh & Cheng-Suang Heng & Zhijie Lin, 2013. "Social Media Brand Community and Consumer Behavior: Quantifying the Relative Impact of User- and Marketer-Generated Content," Information Systems Research, INFORMS, vol. 24(1), pages 88-107, March.
    18. Cary Deck & Erik O. Kimbrough, 2017. "Experimenting with Contests for Experimentation," Southern Economic Journal, John Wiley & Sons, vol. 84(2), pages 391-406, October.
    19. Tran, Trang P., 2017. "Personalized ads on Facebook: An effective marketing tool for online marketers," Journal of Retailing and Consumer Services, Elsevier, vol. 39(C), pages 230-242.
    20. Thomas O. Meservy & Matthew L. Jensen & Kelly J. Fadel, 2014. "Evaluation of Competing Candidate Solutions in Electronic Networks of Practice," Information Systems Research, INFORMS, vol. 25(1), pages 15-34, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:hal:wpaper:halshs-03036837. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: CCSD (email available below). General contact details of provider: https://hal.archives-ouvertes.fr/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.