IDEAS home Printed from https://ideas.repec.org/a/sae/evarev/v40y2016i1p29-60.html
   My bibliography  Save this article

Methodological Credibility

Author

Listed:
  • Miriam R. Jacobson
  • Tarek Azzam

Abstract

Background: When evaluations are broadly disseminated, the public can use them to support a program or to advocate for change. Methods: To explore how evaluations are perceived and used by the public, individuals in a sample of 425 people in the United States were recruited through an online crowdsourcing service called Mechanical Turk ( www.mturk.com ). Participants were randomly assigned to receive different versions of a press release describing a summative evaluation of a program. Each condition contained a unique combination of methods (e.g., randomized controlled design) and findings (positive or negative) to describe the evaluation and its findings. Participants in each condition responded to questions about their trust in the content of the evaluation findings and their attitudes toward the program. Results: Results indicated that the type of evaluation methods and the direction of the findings both influenced the credibility of the findings and that the credibility of the findings moderated the relationship between the direction of the evaluation findings and attitudes toward the evaluated program. Additional evaluation factors to explore in future research with the public are recommended.

Suggested Citation

  • Miriam R. Jacobson & Tarek Azzam, 2016. "Methodological Credibility," Evaluation Review, , vol. 40(1), pages 29-60, February.
    • Handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60
    DOI: 10.1177/0193841X16657728
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0193841X16657728
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0193841X16657728?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Thayer, Colette E. & Fine, Allison H., 2001. "Evaluation and outcome measurement in the non-profit sector: stakeholder participation," Evaluation and Program Planning, Elsevier, vol. 24(1), pages 103-108, February.
    2. Azzam, Tarek & Levine, Bret, 2015. "Politics in evaluation: Politically responsive evaluation in high stakes environments," Evaluation and Program Planning, Elsevier, vol. 53(C), pages 44-56.
    3. Daigneault, Pierre-Marc, 2014. "Taking stock of four decades of quantitative research on stakeholder participation and evaluation use: A systematic map," Evaluation and Program Planning, Elsevier, vol. 45(C), pages 171-181.
    4. Richard T. Carson, 2012. "Contingent Valuation: A Practical Alternative When Prices Aren't Available," Journal of Economic Perspectives, American Economic Association, vol. 26(4), pages 27-42, Fall.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jacobson, Miriam R. & Azzam, Tarek, 2018. "The effects of stakeholder involvement on perceptions of an evaluation’s credibility," Evaluation and Program Planning, Elsevier, vol. 68(C), pages 64-73.
    2. Ivehammar, Pernilla, 2014. "Valuing environmental quality in actual travel time savings – The Haningeleden road project in Stockholm," Research in Transportation Economics, Elsevier, vol. 48(C), pages 349-356.
    3. Lergetporer, Philipp & Schwerdt, Guido & Werner, Katharina & West, Martin R. & Woessmann, Ludger, 2018. "How information affects support for education spending: Evidence from survey experiments in Germany and the United States," Journal of Public Economics, Elsevier, vol. 167(C), pages 138-157.
    4. Cloé Garnache & Scott M. Swinton & Joseph A. Herriges & Frank Lupi & R. Jan Stevenson, 2016. "Solving the Phosphorus Pollution Puzzle: Synthesis and Directions for Future Research," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 98(5), pages 1334-1359.
    5. Eva Wanek & Sacha Bourgeois-Gironde & Alda Mari, 2024. "Desire, moral evaluation or sense of duty: The modal framing of stated preference elicitation," Environmental Values, , vol. 33(4), pages 434-459, August.
    6. Massey, Oliver T., 2011. "A proposed model for the analysis and interpretation of focus groups in evaluation research," Evaluation and Program Planning, Elsevier, vol. 34(1), pages 21-28, February.
    7. Tonin, Stefania, 2018. "Citizens’ perspectives on marine protected areas as a governance strategy to effectively preserve marine ecosystem services and biodiversity," Ecosystem Services, Elsevier, vol. 34(PB), pages 189-200.
    8. Richard T. Carson & Miko_aj Czajkowski, 2014. "The discrete choice experiment approach to environmental contingent valuation," Chapters, in: Stephane Hess & Andrew Daly (ed.), Handbook of Choice Modelling, chapter 9, pages 202-235, Edward Elgar Publishing.
    9. Moeltner, Klaus, 2019. "Bayesian nonlinear meta regression for benefit transfer," Journal of Environmental Economics and Management, Elsevier, vol. 93(C), pages 44-62.
    10. Drichoutis, Andreas C. & Vassilopoulos, Achilleas & Lusk, Jayson L. & Nayga, Rodolfo M. Jr., 2015. "Reference dependence, consequentiality and social desirability in value elicitation: A study of fair labor labeling," 143rd Joint EAAE/AAEA Seminar, March 25-27, 2015, Naples, Italy 202705, European Association of Agricultural Economists.
    11. Bernadeta Gołębiowska & Anna Bartczak & Mikołaj Czajkowski, 2020. "Energy Demand Management and Social Norms," Energies, MDPI, vol. 13(15), pages 1-20, July.
    12. Blomquist, Glenn C. & Coomes, Paul A. & Jepsen, Christopher & Koford, Brandon C. & Troske, Kenneth R., 2014. "Estimating the social value of higher education: willingness to pay for community and technical colleges," Journal of Benefit-Cost Analysis, Cambridge University Press, vol. 5(1), pages 3-41, January.
    13. An Thinh Nguyen & Mai Tran & Thuy Nguyen & Quy Khuc, 2023. "Using Contingent Valuation Method to Explore the Households’ Participation and Willingness to Pay for Improved Plastic Waste Management in North Vietnam," Springer Books, in: An Thinh Nguyen & Thu Thuy Pham & Joon Song & Yen-Ling Lin & Manh Cuong Dong (ed.), Contemporary Economic Issues in Asian Countries: Proceeding of CEIAC 2022, Volume 2, pages 219-237, Springer.
    14. John C. Whitehead, 2024. "Review of Contingent Valuation of Environmental Goods: A Comprehensive Critique. Edited by Daniel McFadden and Kenneth Train (2017): An Update," Working Papers 24-17, Department of Economics, Appalachian State University.
    15. Stefan Eriksson & Per Johansson & Sophie Langenskiöld, 2017. "What is the right profile for getting a job? A stated choice experiment of the recruitment process," Empirical Economics, Springer, vol. 53(2), pages 803-826, September.
    16. Lee, Kyung-Sook & Kim, Ju-Hee & Yoo, Seung-Hoon, 2021. "Would people pay a price premium for electricity from domestic wind power facilities? The case of South Korea," Energy Policy, Elsevier, vol. 156(C).
    17. Drichoutis, Andreas C. & Lusk, Jayson L. & Pappa, Valentina, 2016. "Elicitation formats and the WTA/WTP gap: A study of climate neutral foods," Food Policy, Elsevier, vol. 61(C), pages 141-155.
    18. Lergetporer, P & Woessmann, L, 2022. "Income Contingency and the Electorates Support for Tuition," CAGE Online Working Paper Series 606, Competitive Advantage in the Global Economy (CAGE).
    19. Khanal, Puskar N. & Grebner, Donald L. & Munn, Ian A. & Grado, Stephen C. & Grala, Robert K. & Henderson, James E., 2017. "Evaluating non-industrial private forest landowner willingness to manage for forest carbon sequestration in the southern United States," Forest Policy and Economics, Elsevier, vol. 75(C), pages 112-119.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:evarev:v:40:y:2016:i:1:p:29-60. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.