IDEAS home Printed from https://ideas.repec.org/a/gam/jijerp/v19y2022i22p15262-d977207.html
   My bibliography  Save this article

Inappropriate Evaluation of Effect Modifications Based on Categorical Outcomes: A Systematic Review of Randomized Controlled Trials

Author

Listed:
  • Akihiro Shiroshita

    (Department of Respiratory Medicine, Ichinomiyanishi Hospital, Ichinomiya 494-0001, Japan
    Division of Epidemiology, Department of Medicine, Vanderbilt University School of Medicine, Nashville, TN 37203, USA
    Scientific Research WorkS Peer Support Group (SRWS-PSG), Osaka 541-0043, Japan)

  • Norio Yamamoto

    (Scientific Research WorkS Peer Support Group (SRWS-PSG), Osaka 541-0043, Japan
    Department of Epidemiology, Graduate School of Medicine, Dentistry and Pharmaceutical Sciences, Okayama University, Okayama 700-8558, Japan)

  • Natsumi Saka

    (Scientific Research WorkS Peer Support Group (SRWS-PSG), Osaka 541-0043, Japan
    Department of Health Research Methods, Evidence & Impact, McMaster University, Hamilton, ON L8S 4K1, Canada
    Department of Orthopaedic Surgery, Teikyo University School of Medicine, Tokyo 173-8606, Japan)

  • Motohiro Okumura

    (Department of Neurology, Jikei University School of Medicine, Tokyo 105-8471, Japan)

  • Hiroshi Shiba

    (Department of Internal Medicine, Suwa Central Hospital, Chino 391-8503, Japan)

  • Yuki Kataoka

    (Scientific Research WorkS Peer Support Group (SRWS-PSG), Osaka 541-0043, Japan
    Department of Internal Medicine, Kyoto Min-Iren Asukai Hospital, Kyoto 606-8226, Japan
    Section of Clinical Epidemiology, Department of Community Medicine, Kyoto University Graduate School of Medicine, Kyoto 606-8501, Japan
    Department of Healthcare Epidemiology, Kyoto University Graduate School of Medicine/Public Health, Kyoto 606-8501, Japan)

Abstract

Our meta-epidemiological study aimed to describe the prevalence of reporting effect modification only on relative scale outcomes and inappropriate interpretations of the coefficient of interaction terms in nonlinear models on categorical outcomes. Our study targeted articles published in the top 10 high-impact-factor journals between 1 January and 31 December 2021. We included two-arm, parallel-group, interventional superiority randomized controlled trials to evaluate the effects of modifications on categorical outcomes. The primary outcomes were the prevalence of reporting effect modifications only on relative scale outcomes and that of inappropriately interpreting the coefficient of interaction terms in nonlinear models on categorical outcomes. We included 52 articles, of which 41 (79%) used nonlinear regression to evaluate effect modifications. At least 45/52 articles (87%) reported effect modifications based only on relative scale outcomes, and at least 39/41 (95%) articles inappropriately interpreted the coefficient of interaction terms merely as indices of effect modifications. The quality of the evaluations of effect modifications in nonlinear models on categorical outcomes was relatively low, even in randomized controlled trials published in medical journals with high impact factors. Researchers should report effect modifications of both absolute and relative scale outcomes and avoid interpreting the coefficient of interaction terms in nonlinear regression analyses.

Suggested Citation

  • Akihiro Shiroshita & Norio Yamamoto & Natsumi Saka & Motohiro Okumura & Hiroshi Shiba & Yuki Kataoka, 2022. "Inappropriate Evaluation of Effect Modifications Based on Categorical Outcomes: A Systematic Review of Randomized Controlled Trials," IJERPH, MDPI, vol. 19(22), pages 1-9, November.
    • Handle: RePEc:gam:jijerp:v:19:y:2022:i:22:p:15262-:d:977207
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1660-4601/19/22/15262/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/1660-4601/19/22/15262/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Rainey, Carlisle, 2016. "Compression and Conditional Effects: A Product Term Is Essential When Using Logistic Regression to Test for Interaction," Political Science Research and Methods, Cambridge University Press, vol. 4(3), pages 621-639, September.
    2. Ai, Chunrong & Norton, Edward C., 2003. "Interaction terms in logit and probit models," Economics Letters, Elsevier, vol. 80(1), pages 123-129, July.
    3. Kenneth F Schulz & Douglas G Altman & David Moher & for the CONSORT Group, 2010. "CONSORT 2010 Statement: Updated Guidelines for Reporting Parallel Group Randomised Trials," PLOS Medicine, Public Library of Science, vol. 7(3), pages 1-7, March.
    4. Shengwu Shang & Erik Nesson & Maoyong Fan, 2018. "Interaction Terms In Poisson And Log Linear Regression Models," Bulletin of Economic Research, Wiley Blackwell, vol. 70(1), pages 89-96, January.
    5. Ottar Hellevik, 2009. "Linear versus logistic regression when the dependent variable is a dichotomy," Quality & Quantity: International Journal of Methodology, Springer, vol. 43(1), pages 59-74, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zsuzsa Blasko & Artur Pokropek & Joanna Sikora, 2018. "Science career plans of adolescents: patterns, trends and gender divides," JRC Research Reports JRC109135, Joint Research Centre.
    2. Barbara Dluhosch, 2018. "Trade, Inequality, and Subjective Well-Being: Getting at the Roots of the Backlash Against Globalization," LIS Working papers 741, LIS Cross-National Data Center in Luxembourg.
    3. Xiao, Jing & Lindholm Dahlstrand, Åsa, 2021. "Skill-biased acquisitions? Human capital and target employee mobility in small technology firms," Papers in Innovation Studies 2021/12, Lund University, CIRCLE - Centre for Innovation Research.
    4. Barkowski, Scott, 2021. "Interpretation of nonlinear difference-in-differences: the role of the parallel trends assumption," MPRA Paper 108975, University Library of Munich, Germany.
    5. Jing Xiao & Åsa Lindholm Dahlstrand, 2023. "Skill-biased acquisitions? Human capital and employee mobility in small technology firms," Small Business Economics, Springer, vol. 60(3), pages 1219-1247, March.
    6. Alcott, Benjamin & Rose, Pauline, 2017. "Learning in India’s primary schools: How do disparities widen across the grades?," International Journal of Educational Development, Elsevier, vol. 56(C), pages 42-51.
    7. SHIMIZU Hiroshi & WAKUTSU Naohiko, 2024. "SBIR, Startups, and Subsequent Technological Development: Laser diodes in the United States and Japan," Discussion papers 24012, Research Institute of Economy, Trade and Industry (RIETI).
    8. Mertens, K. & Jacobs, L. & Maes, J. & Poesen, J. & Kervyn, M. & Vranken, L., 2018. "Disaster risk reduction among households exposed to landslide hazard: A crucial role for self-efficacy?," Land Use Policy, Elsevier, vol. 75(C), pages 77-91.
    9. Eelco van der Maat, 2021. "Simplified complexity: Analytical strategies for conflict event research," Conflict Management and Peace Science, Peace Science Society (International), vol. 38(1), pages 87-108, January.
    10. Daoud, Adel & Halleröd, Björn & Guha Sapir, Debarati, 2015. "Quality of government and the relationship between natural disasters and child poverty: A comparative analysis," MPIfG Discussion Paper 15/5, Max Planck Institute for the Study of Societies.
    11. Juho Härkönen, 2011. "Children and Dual Worklessness in Europe: A Comparison of Nine Countries," European Journal of Population, Springer;European Association for Population Studies, vol. 27(2), pages 217-241, May.
    12. Arslan Aziz & Hui Li & Rahul Telang, 2023. "The Consequences of Rating Inflation on Platforms: Evidence from a Quasi-Experiment," Information Systems Research, INFORMS, vol. 34(2), pages 590-608, June.
    13. Ian Peacock & Emily Ryo, 2022. "A study of pandemic and stigma effects in removal proceedings," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 19(3), pages 560-593, September.
    14. Cho, Seo-young & Vadlamannati, Krishna Chaitanya, 2010. "Compliance for big brothers: An empirical analysis on the impact of the anti-trafficking protocol," University of Göttingen Working Papers in Economics 118, University of Goettingen, Department of Economics.
    15. Li, Chen, 2015. "Do immigrants attract FDI? District-level evidence from Germany," VfS Annual Conference 2015 (Muenster): Economic Development - Theory and Policy 113130, Verein für Socialpolitik / German Economic Association.
    16. Abel Brodeur, 2012. "Smoking, Income and Subjective Well-Being: Evidence from Smoking Bans," Working Papers halshs-00664269, HAL.
    17. Chowdhury, Shyamal & Ooi, Evarn & Slonim, Robert, 2017. "Racial discrimination and white first name adoption: a field experiment in the Australian labour market," Working Papers 2017-15, University of Sydney, School of Economics.
    18. Shun-Yang Lee & Julian Runge & Daniel Yoo & Yakov Bart & Anett Gyurak & J. W. Schneider, 2023. "COVID-19 Demand Shocks Revisited: Did Advertising Technology Help Mitigate Adverse Consequences for Small and Midsize Businesses?," Papers 2307.09035, arXiv.org, revised Jan 2024.
    19. Michael J. Fell & Alexandra Schneiders & David Shipworth, 2019. "Consumer Demand for Blockchain-Enabled Peer-to-Peer Electricity Trading in the United Kingdom: An Online Survey Experiment," Energies, MDPI, vol. 12(20), pages 1-25, October.
    20. Cowling, Marc & Ughetto, Elisa & Lee, Neil, 2018. "The innovation debt penalty: Cost of debt, loan default, and the effects of a public loan guarantee on high-tech firms," Technological Forecasting and Social Change, Elsevier, vol. 127(C), pages 166-176.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijerp:v:19:y:2022:i:22:p:15262-:d:977207. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.