IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/xhe3u_v1.html
   My bibliography  Save this paper

How many participants do I need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction

Author

Listed:
  • Sommet, Nicolas
  • Weissman, David Laurence

    (University of Rochester)

  • Cheutin, Nicolas
  • Elliot, Andrew

Abstract

Power analysis for first-order interactions poses two challenges: (a) Conducting an appropriate power analysis is difficult because the typical expected effect size of an interaction depends on its shape, and (b) achieving sufficient power is difficult because interactions are often modest in size. This article consists of three parts. In the first part, we address the first challenge. We first use a fictional study to explain the difference between power analyses for interactions and main effects. Then, we introduce an intuitive taxonomy of 12 types of interactions based on the shape of the interaction (reversed, fully attenuated, partially attenuated) and the size of the simple slopes (median, smaller, larger), and we offer mathematically derived sample-size recommendations to detect each interaction with a power of .80/.90/.95 (for two-tailed tests in between-participants designs). In the second part, we address the second challenge. We first describe a preregistered metastudy (159 studies from recent articles in influential psychology journals) showing that the median power to detect interactions of a typical size is .18. Then, we use simulations (≈900,000,000 data sets) to generate power curves for the 12 types of interactions and test three approaches to increase power without increasing sample size: (a) preregistering one-tailed tests (+21% gain), (b) using a mixed design (+75% gain), and (c) preregistering contrast analysis for a fully attenuated interaction (+62% gain). In the third part, we introduce INT×Power (www.intxpower.com), a web application that enables users to draw their interaction and determine the sample size needed to reach the power of their choice with the option of using/combining these approaches.

Suggested Citation

  • Sommet, Nicolas & Weissman, David Laurence & Cheutin, Nicolas & Elliot, Andrew, 2022. "How many participants do I need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction," OSF Preprints xhe3u_v1, Center for Open Science.
  • Handle: RePEc:osf:osfxxx:xhe3u_v1
    DOI: 10.31219/osf.io/xhe3u_v1
    as

    Download full text from publisher

    File URL: https://osf.io/download/6318ad7909a85221a3d9e41c/
    Download Restriction: no

    File URL: https://libkey.io/10.31219/osf.io/xhe3u_v1?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Majer, Johann M. & Zhang, Kai & Zhang, Hong & Höhne, Benjamin P. & Trötschel, Roman, 2022. "Give and take frames in shared-resource negotiations," Journal of Economic Psychology, Elsevier, vol. 90(C).
    2. Larry V. Hedges & Jacob M. Schauer, 2019. "More Than One Replication Study Is Needed for Unambiguous Tests of Replication," Journal of Educational and Behavioral Statistics, , vol. 44(5), pages 543-570, October.
    3. Wynne W. Chin & Barbara L. Marcolin & Peter R. Newsted, 2003. "A Partial Least Squares Latent Variable Modeling Approach for Measuring Interaction Effects: Results from a Monte Carlo Simulation Study and an Electronic-Mail Emotion/Adoption Study," Information Systems Research, INFORMS, vol. 14(2), pages 189-217, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sommet, Nicolas & Weissman, David Laurence & Cheutin, Nicolas & Elliot, Andrew, 2022. "How many participants do i need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction," OSF Preprints xhe3u, Center for Open Science.
    2. Hazem Ali & Ting Chen & Yunhong Hao, 2021. "Sustainable Manufacturing Practices, Competitive Capabilities, and Sustainable Performance: Moderating Role of Environmental Regulations," Sustainability, MDPI, vol. 13(18), pages 1-19, September.
    3. Claudio Vitari & Elisabetta Raguseo, 2016. "Big data value and financial performance: an empirical investigation [Digital data, dynamic capability and financial performance: an empirical investigation in the era of Big Data]," Post-Print halshs-01923271, HAL.
    4. Mario Silic & Andrea Back, 2016. "The Influence of Risk Factors in Decision-Making Process for Open Source Software Adoption," International Journal of Information Technology & Decision Making (IJITDM), World Scientific Publishing Co. Pte. Ltd., vol. 15(01), pages 151-185, January.
    5. Gupta, Prashant & Seetharaman, A. & Raj, John Rudolph, 2013. "The usage and adoption of cloud computing by small and medium businesses," International Journal of Information Management, Elsevier, vol. 33(5), pages 861-874.
    6. Asif Khan & Chih-Cheng Chen & Kwanrat Suanpong & Athapol Ruangkanjanases & Santhaya Kittikowit & Shih-Chih Chen, 2021. "The Impact of CSR on Sustainable Innovation Ambidexterity: The Mediating Role of Sustainable Supply Chain Management and Second-Order Social Capital," Sustainability, MDPI, vol. 13(21), pages 1-25, November.
    7. Teller, Christoph & Kotzab, Herbert & Grant, David B., 2012. "The relevance of shopper logistics for consumers of store-based retail formats," Journal of Retailing and Consumer Services, Elsevier, vol. 19(1), pages 59-66.
    8. Mehwish Iftikhar & Muhammad Imran Qureshi & Shazia Qayyum & Iram Fatima & Sriyanto Sriyanto & Yasinta Indrianti & Aqeel Khan & Leo-Paul Dana, 2021. "Impact of Multifaceted Workplace Bullying on the Relationships between Technology Usage, Organisational Climate and Employee Physical and Emotional Health," IJERPH, MDPI, vol. 18(6), pages 1-19, March.
    9. M Ashraf Al Haq & Norazlina Abd Wahab & Md. Mahmudul Alam, 2021. "Understanding The Impact Of Institutional Factors On Asnaf Sustainability: A Pls-Sem Approach," Journal of Islamic Monetary Economics and Finance, Bank Indonesia, vol. 7(4), pages 759-790, November.
    10. Osama Sohaib & Kyeong Kang & Mohammad Nurunnabi, 2018. "Gender-Based iTrust in E-Commerce: The Moderating Role of Cognitive Innovativeness," Sustainability, MDPI, vol. 11(1), pages 1-16, December.
    11. Zhang, Chu-Bing & Zhang, Zhuo-Ping & Chang, Ying & Li, Tian-Ge & Hou, Ru-Jing, 2022. "Effect of WeChat interaction on brand evaluation: A moderated mediation model of para-social interaction and affiliative tendency," Journal of Retailing and Consumer Services, Elsevier, vol. 64(C).
    12. Maggie Chuoyan Dong & Yulin Fang & Detmar W. Straub, 2017. "The Impact of Institutional Distance on the Joint Performance of Collaborating Firms: The Role of Adaptive Interorganizational Systems," Information Systems Research, INFORMS, vol. 28(2), pages 309-331, June.
    13. Al-Baraa Abdulrahman Al-Mekhlafi & Idris Othman & Ahmed Farouk Kineber & Ahmad A. Mousa & Ahmad M. A. Zamil, 2022. "Modeling the Impact of Massive Open Online Courses (MOOC) Implementation Factors on Continuance Intention of Students: PLS-SEM Approach," Sustainability, MDPI, vol. 14(9), pages 1-15, April.
    14. Cassandra France & Debra Grace & Joseph Lo Iacono & Joan Carlini, 0. "Exploring the interplay between customer perceived brand value and customer brand co-creation behaviour dimensions," Journal of Brand Management, Palgrave Macmillan, vol. 0, pages 1-15.
    15. Jinjing Yang & Cong Cao & Chensang Ye & Yangyan Shi, 2022. "Effects of Interface Design and Live Atmosphere on Consumers’ Impulse-Buying Behaviour from the Perspective of Human–Computer Interaction," Sustainability, MDPI, vol. 14(12), pages 1-21, June.
    16. Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication success under questionable research practices – a simulation study," MetaArXiv s4b65_v1, Center for Open Science.
    17. María Luisa Pajuelo Moreno & Teresa Duarte-Atoche, 2019. "Relationship between Sustainable Disclosure and Performance—An Extension of Ullmann’s Model," Sustainability, MDPI, vol. 11(16), pages 1-33, August.
    18. Jose Benitez & Laura Ruiz & Ana Castillo & Javier Llorens, 2020. "How corporate social responsibility activities influence employer reputation: The role of social media capability," Post-Print hal-02462583, HAL.
    19. Ascensión Barroso & Óscar R. González-López & Ramón Sanguino & María Buenadicha-Mateos, 2018. "Analysis and Evaluation of the Largest 500 Family Firms’ Websites through PLS-SEM Technique," Sustainability, MDPI, vol. 10(2), pages 1-22, February.
    20. Wei Kang & Qun Wang & Long Cheng & Meng Ning, 2024. "Examining Commuters’ Intention to Use App-Based Carpooling: Insights from the Technology Acceptance Model," Sustainability, MDPI, vol. 16(14), pages 1-17, July.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:xhe3u_v1. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.