IDEAS home Printed from https://ideas.repec.org/p/osf/osfxxx/s5mzp.html
   My bibliography  Save this paper

A Toolbox to Evaluate the Trustworthiness of Published Findings

Author

Listed:
  • Adler, Susanne Jana

    (Ludwig-Maximilians-University Munich)

  • Röseler, Lukas

    (University of Bamberg)

  • Schöniger, Martina Katharina

Abstract

During the past few years, researchers have criticized their professions for providing an entry point for false-positive results arising from publication bias and questionable research practices such as p-hacking (i.e., selectively reporting analyses that yield a p-value below 5 %). Researchers are advocating replication studies and the implementation of open-science practices, like preregistration, in order to identify trustworthy effects. Nevertheless, because such consumer research developments are still emerging, most prior research findings have not been replicated, leaving researchers in the dark as to whether a line of research or a particular effect is trustworthy. We tackle this problem by providing a toolbox containing multiple heuristics to identify data patterns that might, from the information provided in published articles, indicate publication bias and p-hacking. Our toolbox is an easy-to-use instrument with which to initially assess a given set of findings.

Suggested Citation

  • Adler, Susanne Jana & Röseler, Lukas & Schöniger, Martina Katharina, 2023. "A Toolbox to Evaluate the Trustworthiness of Published Findings," OSF Preprints s5mzp, Center for Open Science.
    • Handle: RePEc:osf:osfxxx:s5mzp
    DOI: 10.31219/osf.io/s5mzp
    as

    Download full text from publisher

    File URL: https://osf.io/download/64c53a284f0a4406ac994676/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/s5mzp?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lynch, John G. & Bradlow, Eric T. & Huber, Joel C. & Lehmann, Donald R., 2015. "Reflections on the replication corner: In praise of conceptual replications," International Journal of Research in Marketing, Elsevier, vol. 32(4), pages 333-342.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jun-Hwa Cheah (Jacky) & Francesca Magno & Fabio Cassia, 2024. "Reviewing the SmartPLS 4 software: the latest features and enhancements," Journal of Marketing Analytics, Palgrave Macmillan, vol. 12(1), pages 97-107, March.
    2. Magno, Francesca & Cassia, Fabio, 2024. "Predicting restaurants’ surplus food platform continuance: Insights from the combined use of PLS-SEM and NCA and predictive model comparisons," Journal of Retailing and Consumer Services, Elsevier, vol. 79(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yu Ding & Wayne S. DeSarbo & Dominique M. Hanssens & Kamel Jedidi & John G. Lynch & Donald R. Lehmann, 2020. "The past, present, and future of measurement and methods in marketing analysis," Marketing Letters, Springer, vol. 31(2), pages 175-186, September.
    2. Roy Chen & Yan Chen & Yohanes E. Riyanto, 2021. "Best practices in replication: a case study of common information in coordination games," Experimental Economics, Springer;Economic Science Association, vol. 24(1), pages 2-30, March.
    3. Maya B. Mathur & Tyler J. VanderWeele, 2020. "New statistical metrics for multisite replication projects," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(3), pages 1145-1166, June.
    4. Julia Roloff & Michael J. Zyphur, 2019. "Null Findings, Replications and Preregistered Studies in Business Ethics Research," Journal of Business Ethics, Springer, vol. 160(3), pages 609-619, December.
    5. Robert A. Peterson & U. N. Umesh, 2018. "On the significance of statistically insignificant results in consumer behavior experiments," Journal of the Academy of Marketing Science, Springer, vol. 46(1), pages 81-91, January.
    6. Sheppard, Leah D. & O'Reilly, Jane & van Dijke, Marius & Restubog, Simon Lloyd D. & Aquino, Karl, 2020. "The stress-relieving benefits of positively experienced social sexual behavior in the workplace," Organizational Behavior and Human Decision Processes, Elsevier, vol. 156(C), pages 38-52.
    7. Gleibs, Ilka H. & Lizama Alvarado, Andrea, 2019. "The impact of non-standard work arrangements and communication climate on organisational and team identification and work-related outcomes amongst millennial in Chile and the UK," LSE Research Online Documents on Economics 101504, London School of Economics and Political Science, LSE Library.
    8. Mathur, Maya B & VanderWeele, Tyler, 2018. "Statistical methods for evidence synthesis," Thesis Commons kd6ja, Center for Open Science.
    9. Nicolas G. A. Lorgnier & Che-Jen Su & Shawn M. O’Rourke, 2022. "Brands’ perceived sustainable development goals: index development and applications with professional sport teams and fast-food brands," Service Business, Springer;Pan-Pacific Business Association, vol. 16(1), pages 125-157, March.
    10. Barbarossa, Camilla & Buzeta, Cristian & De Pelsmacker, Patrick & Moons, Ingrid, 2022. "Foreign company misconduct and how consumers’ punitive intent is influenced by country stereotypes and the perceived similarity between the foreign country and the home country," International Business Review, Elsevier, vol. 31(5).
    11. Diamantopoulos, Adamantios & Matarazzo, Michela & Montanari, Maria Gabriela & Petrychenko, Anastasiya, 2021. "The “Pricing Footprint” of Country-of-Origin: Conceptualization and Empirical Assessment," Journal of Business Research, Elsevier, vol. 135(C), pages 749-757.
    12. Kienzler, Mario & Kowalkowski, Christian & Kindström, Daniel, 2021. "Purchasing professionals and the flat-rate bias: Effects of price premiums, past usage, and relational ties on price plan choice," Journal of Business Research, Elsevier, vol. 132(C), pages 403-415.
    13. Adler, Susanne Jana & Röseler, Lukas & Schöniger, Martina Katharina, 2023. "A toolbox to evaluate the trustworthiness of published findings," Journal of Business Research, Elsevier, vol. 167(C).
    14. Liu, Yunxin & Dewitte, Siegfried, 2021. "A replication study of the credit card effect on spending behavior and an extension to mobile payments," Journal of Retailing and Consumer Services, Elsevier, vol. 60(C).
    15. Adler, Susanne Jana & Sharma, Pratyush Nidhi & Radomir, Lăcrămioara, 2023. "Toward open science in PLS-SEM: Assessing the state of the art and future perspectives," Journal of Business Research, Elsevier, vol. 169(C).
    16. Swani, Kunal & Milne, George R. & Slepchuk, Alec N., 2021. "Revisiting Trust and Privacy Concern in Consumers' Perceptions of Marketing Information Management Practices: Replication and Extension," Journal of Interactive Marketing, Elsevier, vol. 56(C), pages 137-158.
    17. Geoffrey Fisher & Matthew McGranaghan & Jura Liaukonyte & Kenneth C. Wilbur, 2023. "Price promotions, beneficiary framing, and mental accounting," Quantitative Marketing and Economics (QME), Springer, vol. 21(2), pages 147-181, June.
    18. Lehmann, Donald R., 2020. "The evolving world of research in marketing and the blending of theory and data," International Journal of Research in Marketing, Elsevier, vol. 37(1), pages 27-42.
    19. Maier, Erik & Dost, Florian, 2018. "Fluent contextual image backgrounds enhance mental imagery and evaluations of experience products," Journal of Retailing and Consumer Services, Elsevier, vol. 45(C), pages 207-220.
    20. Karl Akbari & Udo Wagner, 2021. "Playing When Paying and What Happens Next: Customer Satisfaction and Word-of-Mouth Intention in Gambled Price Promotions," Schmalenbach Journal of Business Research, Springer, vol. 73(2), pages 243-271, June.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:osfxxx:s5mzp. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.