IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/xhe3u.html
   My bibliography  Save this paper

How many participants do i need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction

Author

Listed:
  • Sommet, Nicolas
  • Weissman, David Laurence

    (University of Rochester)

  • Cheutin, Nicolas
  • Elliot, Andrew

Abstract

Power analysis for first-order interactions poses two challenges: (i) Conducting an appropriate power analysis is difficult, because the required sample size depends on the shape of the interaction and the size of the simple slopes; (ii) Achieving sufficient power is difficult, because interactions tend to be modest in size. This paper consists of three parts. PART 1 addresses the first challenge. We first describe a fictional study to illustrate in a simple way how power analyses differ between interactions and main effects. Then, we introduce an intuitive taxonomy of 12 types of interactions based on the shape of the interaction (reversed, fully attenuated, partially attenuated) and the size of the simple effects (small, medium, large), and we offer mathematically-derived sample size recommendations to detect each interaction with a power of .80/.90. PART 2 addresses the second challenge. We first describe a preregistered meta-study (159 studies from recent articles in influential psychology journals) showing that the median power to detect medium-sized interactions in the literature is .18. Then, we use simulations (887M simulated datasets) to generate power curves for the 12 types of interactions, and test three strategies to increase power without increasing sample size: (i) preregistering one-tailed tests (+21% gain), (ii) using a mixed design (+75% gain), and (ii) preregistering contrast analysis for a fully attenuated interaction (+63% gain). PART 3 introduces INT×Power, a web-application that enables users to draw their interaction and determine the sample size needed to reach a power of .80 with and without using these strategies: www.intxpower.com.

Suggested Citation

  • Sommet, Nicolas & Weissman, David Laurence & Cheutin, Nicolas & Elliot, Andrew, 2022. "How many participants do i need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction," OSF Preprints xhe3u, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:xhe3u
    DOI: 10.31219/osf.io/xhe3u
    as

    Download full text from publisher

    File URL: https://osf.io/download/6318ad7909a85221a3d9e41c/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/xhe3u?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Majer, Johann M. & Zhang, Kai & Zhang, Hong & Höhne, Benjamin P. & Trötschel, Roman, 2022. "Give and take frames in shared-resource negotiations," Journal of Economic Psychology, Elsevier, vol. 90(C).
    2. Larry V. Hedges & Jacob M. Schauer, 2019. "More Than One Replication Study Is Needed for Unambiguous Tests of Replication," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 543-570, October.
    3. Wynne W. Chin & Barbara L. Marcolin & Peter R. Newsted, 2003. "A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic-Mail Emotion/Adoption Study," Information Systems Research, INFORMS, vol. 14(2), pages 189-217, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hazem Ali & Ting Chen & Yunhong Hao, 2021. "Sustainable Manufacturing Practices, Competitive Capabilities, and Sustainable Performance: Moderating Role of Environmental Regulations," Sustainability, MDPI, vol. 13(18), pages 1-19, September.
    2. Claudio Vitari & Elisabetta Raguseo, 2016. "Big data value and financial performance: an empirical investigation [Digital data, dynamic capability and financial performance: an empirical investigation in the era of Big Data]," Post-Print halshs-01923271, HAL.
    3. Mario Silic & Andrea Back, 2016. "The Influence of Risk Factors in Decision-Making Process for Open Source Software Adoption," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(01), pages 151-185, January.
    4. Gupta, Prashant & Seetharaman, A. & Raj, John Rudolph, 2013. "The usage and adoption of cloud computing by small and medium businesses," International Journal of Information Management, Elsevier, vol. 33(5), pages 861-874.
    5. Asif Khan & Chih-Cheng Chen & Kwanrat Suanpong & Athapol Ruangkanjanases & Santhaya Kittikowit & Shih-Chih Chen, 2021. "The Impact of CSR on Sustainable Innovation Ambidexterity: The Mediating Role of Sustainable Supply Chain Management and Second-Order Social Capital," Sustainability, MDPI, vol. 13(21), pages 1-25, November.
    6. Teller, Christoph & Kotzab, Herbert & Grant, David B., 2012. "The relevance of shopper logistics for consumers of store-based retail formats," Journal of Retailing and Consumer Services, Elsevier, vol. 19(1), pages 59-66.
    7. Mehwish Iftikhar & Muhammad Imran Qureshi & Shazia Qayyum & Iram Fatima & Sriyanto Sriyanto & Yasinta Indrianti & Aqeel Khan & Leo-Paul Dana, 2021. "Impact of Multifaceted Workplace Bullying on the Relationships between Technology Usage, Organisational Climate and Employee Physical and Emotional Health," IJERPH, MDPI, vol. 18(6), pages 1-19, March.
    8. M Ashraf Al Haq & Norazlina Abd Wahab & Md. Mahmudul Alam, 2021. "Understanding The Impact Of Institutional Factors On Asnaf Sustainability: A Pls-Sem Approach," Journal of Islamic Monetary Economics and Finance, Bank Indonesia, vol. 7(4), pages 759-790, November.
    9. Osama Sohaib & Kyeong Kang & Mohammad Nurunnabi, 2018. "Gender-Based iTrust in E-Commerce: The Moderating Role of Cognitive Innovativeness," Sustainability, MDPI, vol. 11(1), pages 1-16, December.
    10. Zhang, Chu-Bing & Zhang, Zhuo-Ping & Chang, Ying & Li, Tian-Ge & Hou, Ru-Jing, 2022. "Effect of WeChat interaction on brand evaluation: A moderated mediation model of para-social interaction and affiliative tendency," Journal of Retailing and Consumer Services, Elsevier, vol. 64(C).
    11. Maggie Chuoyan Dong & Yulin Fang & Detmar W. Straub, 2017. "The Impact of Institutional Distance on the Joint Performance of Collaborating Firms: The Role of Adaptive Interorganizational Systems," Information Systems Research, INFORMS, vol. 28(2), pages 309-331, June.
    12. Al-Baraa Abdulrahman Al-Mekhlafi & Idris Othman & Ahmed Farouk Kineber & Ahmad A. Mousa & Ahmad M. A. Zamil, 2022. "Modeling the Impact of Massive Open Online Courses (MOOC) Implementation Factors on Continuance Intention of Students: PLS-SEM Approach," Sustainability, MDPI, vol. 14(9), pages 1-15, April.
    13. Cassandra France & Debra Grace & Joseph Lo Iacono & Joan Carlini, 0. "Exploring the interplay between customer perceived brand value and customer brand co-creation behaviour dimensions," Journal of Brand Management, Palgrave Macmillan, vol. 0, pages 1-15.
    14. Jinjing Yang & Cong Cao & Chensang Ye & Yangyan Shi, 2022. "Effects of Interface Design and Live Atmosphere on Consumers’ Impulse-Buying Behaviour from the Perspective of Human–Computer Interaction," Sustainability, MDPI, vol. 14(12), pages 1-21, June.
    15. María Luisa Pajuelo Moreno & Teresa Duarte-Atoche, 2019. "Relationship between Sustainable Disclosure and Performance—An Extension of Ullmann’s Model," Sustainability, MDPI, vol. 11(16), pages 1-33, August.
    16. Jose Benitez & Laura Ruiz & Ana Castillo & Javier Llorens, 2020. "How corporate social responsibility activities influence employer reputation: The role of social media capability," Post-Print hal-02462583, HAL.
    17. Ascensión Barroso & Óscar R. González-López & Ramón Sanguino & María Buenadicha-Mateos, 2018. "Analysis and Evaluation of the Largest 500 Family Firms’ Websites through PLS-SEM Technique," Sustainability, MDPI, vol. 10(2), pages 1-22, February.
    18. Wei Kang & Qun Wang & Long Cheng & Meng Ning, 2024. "Examining Commuters’ Intention to Use App-Based Carpooling: Insights from the Technology Acceptance Model," Sustainability, MDPI, vol. 16(14), pages 1-17, July.
    19. Chaeyoung Lim & Jongchang Ahn, 2021. "Social Overload and Discontinuance Intention on Facebook: A Comparative Study," Sustainability, MDPI, vol. 13(22), pages 1-17, November.
    20. Rui Guo & Xiaoying Liu & Hakjun Song, 2021. "Structural Relationships among Strategic Experiential Modules, Motivation, Serious Leisure, Satisfaction and Quality of Life in Bicycle Tourism," IJERPH, MDPI, vol. 18(23), pages 1-13, December.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:xhe3u. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.