IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v56y2022i5d10.1007_s11135-021-01250-3.html
   My bibliography  Save this article

Can we trust measures of trust? a comparison of results from open and closed questions

Author

Listed:
  • Anna Brosius

    (Amsterdam School of Communication Research, University of Amsterdam)

  • Michael Hameleers

    (Amsterdam School of Communication Research, University of Amsterdam)

  • Toni G. L. A. Meer

    (Amsterdam School of Communication Research, University of Amsterdam)

Abstract

Many public opinion surveys compare trust in a number of different information and (mediated) knowledge sources, typically using closed questions with a set of answer categories that are imposed by researchers. We aim to validate these categories by quantitatively comparing survey responses about trustworthy sources using open and closed questions, and by qualitatively analyzing the open answers. The results show that answer options typically used for closed questions in academic research are generally valid and closely match categories that respondents come up with unprimed. In some cases, answers to open questions can be non-exhaustive, particularly when sources are considered trustworthy but are not salient for respondents. Open questions, however, may still be useful for exploratory research or more detailed investigations of media diets on the outlet-level. Qualitative approaches to open questions can also give more insight into motivations for distrust, e.g. perceptions of inconsistency or a fundamental rejection of the shared factual basis of an issue. In addition, our results indicate that respondents’ interpretation of answer categories may change reported levels of trust: those that think of more specific outlets tend to report higher general media trust. This study provides new insights into how question design, and particularly the choice of answer options, may influence reported levels and sources of trust, and how qualitative and quantitative approaches to trust measurement can be combined.

Suggested Citation

  • Anna Brosius & Michael Hameleers & Toni G. L. A. Meer, 2022. "Can we trust measures of trust? a comparison of results from open and closed questions," Quality & Quantity: International Journal of Methodology, Springer, vol. 56(5), pages 2907-2924, October.
  • Handle: RePEc:spr:qualqt:v:56:y:2022:i:5:d:10.1007_s11135-021-01250-3
    DOI: 10.1007/s11135-021-01250-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-021-01250-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.

    File URL: https://libkey.io/10.1007/s11135-021-01250-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mueller, John E., 1970. "Presidential Popularity from Truman to Johnson1," American Political Science Review, Cambridge University Press, vol. 64(1), pages 18-34, March.
    2. Blair, Robert A. & Morse, Benjamin S. & Tsai, Lily L., 2017. "Public health and public trust: Survey evidence from the Ebola Virus Disease epidemic in Liberia," Social Science & Medicine, Elsevier, vol. 172(C), pages 89-97.
    3. Anna Brosius & Erika J van Elsas & Claes H de Vreese, 2020. "Trust in context: National heuristics and survey context effects on political trust in the European Union," European Union Politics, , vol. 21(2), pages 294-311, June.
    4. Mary Jo Kealy & Robert W. Turner, 1993. "A Test of the Equality of Closed-Ended and Open-Ended Contingent Valuations," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 75(2), pages 321-331.
    5. Hetherington, Marc J., 1998. "The Political Relevance of Political Trust," American Political Science Review, Cambridge University Press, vol. 92(4), pages 791-808, December.
    6. Oddgeir Friborg & Jan Rosenvinge, 2013. "A comparison of open-ended and closed questions in the prediction of mental health," Quality & Quantity: International Journal of Methodology, Springer, vol. 47(3), pages 1397-1411, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Mateusz Borkowski, 2024. "Trust and economic development on the example of European economies in 2017–2020: PLS-SEM modeling," Quality & Quantity: International Journal of Methodology, Springer, vol. 58(5), pages 4257-4280, October.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Barry Eichengreen & Orkun Saka & Cevat Giray Aksoy, 2024. "The Political Scar of Epidemics," The Economic Journal, Royal Economic Society, vol. 134(660), pages 1683-1700.
    2. Nicholas Biddle & Matthew Gray & Ian McAllister, 2024. "Federalism and Confidence in Australian Governments During the COVID-19 Pandemic," Publius: The Journal of Federalism, CSF Associates Inc., vol. 54(2), pages 257-282.
    3. repec:zbw:bofitp:2020_014 is not listed on IDEAS
    4. Tianjing Liao, 2023. "State international income position, trade openness, and mass antigovernment protests," International Area Studies Review, Center for International Area Studies, Hankuk University of Foreign Studies, vol. 26(4), pages 402-418, December.
    5. Aksoy, Cevat Giray & Eichengreen, Barry & Saka, Orkun, 2020. "The Political Scar of Epidemics," IZA Discussion Papers 13351, Institute of Labor Economics (IZA).
    6. Ben Davies & Fanny Lalot & Linus Peitz & Maria S. Heering & Hilal Ozkececi & Jacinta Babaian & Kaya Davies Hayon & Jo Broadwood & Dominic Abrams, 2021. "Changes in political trust in Britain during the COVID-19 pandemic in 2020: integrated public opinion evidence and implications," Palgrave Communications, Palgrave Macmillan, vol. 8(1), pages 1-9, December.
    7. Collewet, Marion & Fairley, Kim & Kessels, Roselinde & Knoef, Marike & van Vliet, Olaf, 2024. "The design of welfare: unraveling taxpayers' preferences," OSF Preprints 4am7e, Center for Open Science.
    8. Jaeyoung Lim & Kuk-Kyoung Moon, 2021. "Can Political Trust Weaken the Relationship between Perceived Environmental Threats and Perceived Nuclear Threats? Evidence from South Korea," IJERPH, MDPI, vol. 18(18), pages 1-13, September.
    9. repec:zbw:bofitp:2022_009 is not listed on IDEAS
    10. Rodrigo Martins & Francisco Veiga, 2013. "Economic voting in Portuguese municipal elections," Public Choice, Springer, vol. 155(3), pages 317-334, June.
    11. Nils D. Steiner & Ruxanda Berlinschi & Etienne Farvaque & Jan Fidrmuc & Philipp Harms & Alexander Mihailov & Michael Neugart & Piotr Stanek, 2023. "Rallying around the EU flag: Russia's invasion of Ukraine and attitudes toward European integration," Journal of Common Market Studies, Wiley Blackwell, vol. 61(2), pages 283-301, March.
    12. Hurlimann, Anna C., 2009. "Water supply in regional Victoria Australia: A review of the water cartage industry and willingness to pay for recycled water," Resources, Conservation & Recycling, Elsevier, vol. 53(5), pages 262-268.
    13. Antoine Auberger, 2011. "Popularity Functions for the French President and Prime Minister (1995-2007)," Working Papers halshs-00872313, HAL.
    14. Toke S. Aidt & Francisco José Veiga & Linda Gonçalves Veiga, 2007. "Election Results and Opportunistic Policies: An Integrated Approach," NIPE Working Papers 24/2007, NIPE - Universidade do Minho.
    15. Yann Algan & Pierre Cahuc & Marc Sangnier, 2016. "Trust and the Welfare State: the Twin Peaks Curve," Economic Journal, Royal Economic Society, vol. 126(593), pages 861-883, June.
    16. Henderson, James E. & Dunn, Michael A., 2007. "Investigating the Potential of Fee-Based Recreation on Private Lands in the Lower Mississippi River Delta," 2007 Annual Meeting, February 4-7, 2007, Mobile, Alabama 34822, Southern Agricultural Economics Association.
    17. Stéfanie André, 2014. "Does Trust Mean the Same for Migrants and Natives? Testing Measurement Models of Political Trust with Multi-group Confirmatory Factor Analysis," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 115(3), pages 963-982, February.
    18. Rinaldo Brau & Gianluca Fiorentini & Matteo Lippi Bruni & Anna Maria Pinna, 2004. "La disponibilità a pagare per la copertura del rischio di non autosufficienza: analisi econometrica e valutazioni di "policy"," Politica economica, Società editrice il Mulino, issue 3, pages 357-388.
    19. Kellner, Adrian & Martinussen, Pål Erling & Feiring, Eli, 2023. "Don't stand so close to me: Perceptions of others’ compliance with COVID-19 recommendations and support for strict policy measures in Norway," Health Policy, Elsevier, vol. 136(C).
    20. Teresa Cuerdo-Vilches & Miguel Ángel Navas-Martín & Ignacio Oteiza, 2020. "A Mixed Approach on Resilience of Spanish Dwellings and Households during COVID-19 Lockdown," Sustainability, MDPI, vol. 12(23), pages 1-24, December.
    21. Nunkoo, Robin & Smith, Stephen L.J., 2013. "Political economy of tourism: Trust in government actors, political support, and their determinants," Tourism Management, Elsevier, vol. 36(C), pages 120-132.
    22. Kyle Haynes, 2017. "Diversionary conflict: Demonizing enemies or demonstrating competence?," Conflict Management and Peace Science, Peace Science Society (International), vol. 34(4), pages 337-358, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:56:y:2022:i:5:d:10.1007_s11135-021-01250-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.