IDEAS home Printed from https://ideas.repec.org/a/eee/epplan/v98y2023ics0149718923000514.html
   My bibliography  Save this article

Consequences of survey modification in a program evaluation: An exploratory research on evaluation study

Author

Listed:
  • Braman, Colin
  • Azzam, Tarek

Abstract

The purpose of this exploratory research on evaluation study was to examine how modified and unmodified scales of critical thinking and interest in science careers would affect the evaluation conclusions. Surveys measuring various outcomes that are used in program evaluation are frequently modified from their original versions in response to the unique context of programs. Modifying existing published surveys by removing or adding items can affect the psychometric properties of the original scale and may produce differing results. The results of the comparisons found that unmodified and modified surveys had similar reliabilities; however, one of the scales produced contradictory evaluation findings. Lessons learned from this study suggest that scales can be modified in evaluation, but great care is needed to address the potential strengths and limitations of the modified scale and balance the technical needs with responsiveness to program context.

Suggested Citation

  • Braman, Colin & Azzam, Tarek, 2023. "Consequences of survey modification in a program evaluation: An exploratory research on evaluation study," Evaluation and Program Planning, Elsevier, vol. 98(C).
    • Handle: RePEc:eee:epplan:v:98:y:2023:i:c:s0149718923000514
    DOI: 10.1016/j.evalprogplan.2023.102274
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0149718923000514
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.evalprogplan.2023.102274?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Greene, Jennifer C., 1990. "Technical quality versus user responsiveness in evaluation practice," Evaluation and Program Planning, Elsevier, vol. 13(3), pages 267-274, January.
    2. Flynn, Leisa Reinecke & Goldsmith, Ronald E., 1999. "A Short, Reliable Measure of Subjective Knowledge," Journal of Business Research, Elsevier, vol. 46(1), pages 57-66, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Groza, Mark D. & Groza, Mya Pronschinske, 2018. "Salesperson regulatory knowledge and sales performance," Journal of Business Research, Elsevier, vol. 89(C), pages 37-46.
    2. Lim, Thien Sang & Mail, Rasid & Abd Karim, Mohd Rahimie & Ahmad Baharul Ulum, Zatul Karamah & Jaidi, Junainah & Noordin, Raman, 2018. "A serial mediation model of financial knowledge on the intention to invest: The central role of risk perception and attitude," Journal of Behavioral and Experimental Finance, Elsevier, vol. 20(C), pages 74-79.
    3. de Lauwere, Carolien & Slegers, Monique & Meeusen, Marieke, 2022. "The influence of behavioural factors and external conditions on Dutch farmers’ decision making in the transition towards circular agriculture," Land Use Policy, Elsevier, vol. 120(C).
    4. Catherine Viot & Juliette Passebois-Ducros, 2010. "Wine brands or branded wines? The specificity of the French market in terms of the brand," Post-Print hal-01803728, HAL.
    5. Carla S. Marques & Carlos P. Marques & João J. M. Ferreira & Fernando A. F. Ferreira, 2019. "Effects of traits, self-motivation and managerial skills on nursing intrapreneurship," International Entrepreneurship and Management Journal, Springer, vol. 15(3), pages 733-748, September.
    6. Eva M. Murgado-Armenteros & María Gutierrez-Salcedo & Francisco José Torres-Ruiz, 2020. "The Concern about Biodiversity as a Criterion for the Classification of the Sustainable Consumer: A Cross-Cultural Approach," Sustainability, MDPI, vol. 12(8), pages 1-14, April.
    7. Benedikt Berger & Martin Adam & Alexander Rühr & Alexander Benlian, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(1), pages 55-68, February.
    8. Feucht, Yvonne & Zander, Katrin, 2017. "Consumers’ attitudes on carbon footprint labelling. Results of the SUSDIET project," Thünen Working Paper 266396, Johann Heinrich von Thünen-Institut (vTI), Federal Research Institute for Rural Areas, Forestry and Fisheries.
    9. Kwon, Kyoung-Nan & Lee, Jinkook, 2009. "The effects of reference point, knowledge, and risk propensity on the evaluation of financial products," Journal of Business Research, Elsevier, vol. 62(7), pages 719-725, July.
    10. Hatice Aydýn & Cemal Zehir, 2017. "What Type Relationship Do We Have with Our Brands? Is the Name of this Relationship Brand Romance?," International Review of Management and Marketing, Econjournals, vol. 7(2), pages 272-283.
    11. Groza, Mark D. & Locander, David A. & Howlett, Charles H., 2016. "Linking thinking styles to sales performance: The importance of creativity and subjective knowledge," Journal of Business Research, Elsevier, vol. 69(10), pages 4185-4193.
    12. Casenave, Eric & Klarmann, Martin, 2020. "The accountability paradox: How holding marketers accountable hinders alignment with short-term marketing goals," Journal of Business Research, Elsevier, vol. 112(C), pages 95-108.
    13. Song, Lei & Meng, Yan & Chang, Hua & Li, Wenjing & Tan (Frank), Kang, 2021. "How counterfeit dominance affects luxury fashion brand owners’ perceptions: A cross-cultural examination," Journal of Business Research, Elsevier, vol. 130(C), pages 1-13.
    14. Hyun Joo Kwon & Mira Ahn & Jiyun Kang, 2021. "The Effects of Knowledge Types on Consumer Decision Making for Non-Toxic Housing Materials and Products," Sustainability, MDPI, vol. 13(19), pages 1-14, October.
    15. Sven Gruener, 2024. "Determinants of Gullibility to Misinformation: A Study of Climate Change, COVID-19 and Artificial Intelligence," Journal of Interdisciplinary Economics, , vol. 36(1), pages 58-78, January.
    16. María del Mar Serrano-Arcos & Raquel Sánchez-Fernández & Juan Carlos Pérez-Mesa, 2021. "Analysis of Product-Country Image from Consumer’s Perspective: The Impact of Subjective Knowledge, Perceived Risk and Media Influence," Sustainability, MDPI, vol. 13(4), pages 1-25, February.
    17. Daunt, Kate L. & Harris, Lloyd C., 2017. "Consumer showrooming: Value co-destruction," Journal of Retailing and Consumer Services, Elsevier, vol. 38(C), pages 166-176.
    18. Christian Hildebrand & Gerald Häubl & Andreas Herrmann & Jan R. Landwehr, 2013. "When Social Media Can Be Bad for You: Community Feedback Stifles Consumer Creativity and Reduces Satisfaction with Self-Designed Products," Information Systems Research, INFORMS, vol. 24(1), pages 14-29, March.
    19. Wei, Wei & Kim, Gaeul & Miao, Li & Behnke, Carl & Almanza, Barbara, 2018. "Consumer inferences of corporate social responsibility (CSR) claims on packaged foods," Journal of Business Research, Elsevier, vol. 83(C), pages 186-201.
    20. Hyun Joung Jin & Dae Hee Han, 2019. "College Students’ Experience of a Food Safety Class and Their Responses to the MSG Issue," IJERPH, MDPI, vol. 16(16), pages 1-13, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:epplan:v:98:y:2023:i:c:s0149718923000514. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/evalprogplan .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.