IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v40y2016i1p29-60.html
   My bibliography  Save this article

Methodological Credibility

Author

Listed:
  • Miriam R. Jacobson
  • Tarek Azzam

Abstract

Background: When evaluations are broadly disseminated, the public can use them to support a program or to advocate for change. Methods: To explore how evaluations are perceived and used by the public, individuals in a sample of 425 people in the United States were recruited through an online crowdsourcing service called Mechanical Turk ( www.mturk.com ). Participants were randomly assigned to receive different versions of a press release describing a summative evaluation of a program. Each condition contained a unique combination of methods (e.g., randomized controlled design) and findings (positive or negative) to describe the evaluation and its findings. Participants in each condition responded to questions about their trust in the content of the evaluation findings and their attitudes toward the program. Results: Results indicated that the type of evaluation methods and the direction of the findings both influenced the credibility of the findings and that the credibility of the findings moderated the relationship between the direction of the evaluation findings and attitudes toward the evaluated program. Additional evaluation factors to explore in future research with the public are recommended.

Suggested Citation

  • Miriam R. Jacobson & Tarek Azzam, 2016. "Methodological Credibility," Evaluation Review, , vol. 40(1), pages 29-60, February.
    • Handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60
    DOI: 10.1177/0193841X16657728
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X16657728
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X16657728?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Daigneault, Pierre-Marc, 2014. "Taking stock of four decades of quantitative research on stakeholder participation and evaluation use: A systematic map," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 171-181.
    2. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    3. Thayer, Colette E. & Fine, Allison H., 2001. "Evaluation and outcome measurement in the non-profit sector: stakeholder participation," Evaluation and Program Planning, Elsevier, vol. 24(1), pages 103-108, February.
    4. Azzam, Tarek & Levine, Bret, 2015. "Politics in evaluation: Politically responsive evaluation in high stakes environments," Evaluation and Program Planning, Elsevier, vol. 53(C), pages 44-56.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jacobson, Miriam R. & Azzam, Tarek, 2018. "The effects of stakeholder involvement on perceptions of an evaluation’s credibility," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 64-73.
    2. Ivehammar, Pernilla, 2014. "Valuing environmental quality in actual travel time savings – The Haningeleden road project in Stockholm," Research in Transportation Economics, Elsevier, vol. 48(C), pages 349-356.
    3. Massey, Oliver T., 2011. "A proposed model for the analysis and interpretation of focus groups in evaluation research," Evaluation and Program Planning, Elsevier, vol. 34(1), pages 21-28, February.
    4. Tonin, Stefania, 2018. "Citizens’ perspectives on marine protected areas as a governance strategy to effectively preserve marine ecosystem services and biodiversity," Ecosystem Services, Elsevier, vol. 34(PB), pages 189-200.
    5. Moeltner, Klaus, 2019. "Bayesian nonlinear meta regression for benefit transfer," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 44-62.
    6. Bernadeta Gołębiowska & Anna Bartczak & Mikołaj Czajkowski, 2020. "Energy Demand Management and Social Norms," Energies, MDPI, vol. 13(15), pages 1-20, July.
    7. Blomquist, Glenn C. & Coomes, Paul A. & Jepsen, Christopher & Koford, Brandon C. & Troske, Kenneth R., 2014. "Estimating the social value of higher education: willingness to pay for community and technical colleges," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 5(1), pages 3-41, January.
    8. An Thinh Nguyen & Mai Tran & Thuy Nguyen & Quy Khuc, 2023. "Using Contingent Valuation Method to Explore the Households’ Participation and Willingness to Pay for Improved Plastic Waste Management in North Vietnam," Springer Books, in: An Thinh Nguyen & Thu Thuy Pham & Joon Song & Yen-Ling Lin & Manh Cuong Dong (ed.), Contemporary Economic Issues in Asian Countries: Proceeding of CEIAC 2022, Volume 2, pages 219-237, Springer.
    9. Stefan Eriksson & Per Johansson & Sophie Langenskiöld, 2017. "What is the right profile for getting a job? A stated choice experiment of the recruitment process," Empirical Economics, Springer, vol. 53(2), pages 803-826, September.
    10. Lee, Kyung-Sook & Kim, Ju-Hee & Yoo, Seung-Hoon, 2021. "Would people pay a price premium for electricity from domestic wind power facilities? The case of South Korea," Energy Policy, Elsevier, vol. 156(C).
    11. Desvousges, William H. & Gard, Nicholas & Michael, Holly J. & Chance, Anne D., 2018. "Habitat and Resource Equivalency Analysis: A Critical Assessment," Ecological Economics, Elsevier, vol. 143(C), pages 74-89.
    12. Donald S. Kenkel & Sida Peng & Michael F. Pesko & Hua Wang, 2020. "Mostly harmless regulation? Electronic cigarettes, public policy, and consumer welfare," Health Economics, John Wiley & Sons, Ltd., vol. 29(11), pages 1364-1377, November.
    13. Daniel R. Petrolia & Matthew G. Interis & Joonghyun Hwang, 2018. "Single-Choice, Repeated-Choice, and Best-Worst Scaling Elicitation Formats: Do Results Differ and by How Much?," Environmental & Resource Economics, Springer;European Association of Environmental and Resource Economists, vol. 69(2), pages 365-393, February.
    14. Sobolewski, Maciej & Czajkowski, Mikołaj, 2018. "Receiver benefits and strategic use of call externalities in mobile telephony markets," Information Economics and Policy, Elsevier, vol. 44(C), pages 16-27.
    15. Ahlheim, Michael & Schneider, Friedrich, 2013. "Considering Household Size in Contingent Valuation Studies," VfS Annual Conference 2013 (Duesseldorf): Competition Policy and Regulation in a Global Economic Order 79974, Verein für Socialpolitik / German Economic Association.
    16. Aziz, Sonia & Pakhtigian, Emily L. & Akanda, Ali S. & Jutla, Antarpreet & Huq, Anwar & Alam, Munirul & Ashan, Gias U. & Colwell, Rita R., 2021. "Does improved risk information increase the value of cholera prevention? An analysis of stated vaccine demand in slum areas of urban Bangladesh," Social Science & Medicine, Elsevier, vol. 272(C).
    17. Lergetporer, Philipp & Woessmann, Ludger, 2023. "Earnings information and public preferences for university tuition: Evidence from representative experiments," Journal of Public Economics, Elsevier, vol. 226(C).
    18. Laia Soler & Nicolas Borzykowski, 2021. "The costs of celiac disease: a contingent valuation in Switzerland," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 22(9), pages 1487-1505, December.
    19. DECANCQ, Koen & FLEURBAEY, Marc & SCHOKKAERT, Erik, 2014. "Inequality, income, and well-being," LIDAM Discussion Papers CORE 2014018, Université catholique de Louvain, Center for Operations Research and Econometrics (CORE).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.