IDEAS home Printed from https://ideas.repec.org/a/spr/ariqol/v14y2019i2d10.1007_s11482-018-9628-x.html
   My bibliography  Save this article

The Effects of Features of Survey Measurement on Self-Rated Health: Response Option Order and Scale Orientation

Author

Listed:
  • Dana Garbarski

    (Loyola University Chicago)

  • Nora Cate Schaeffer

    (University of Wisconsin-Madison
    University of Wisconsin-Madison)

  • Jennifer Dykema

    (University of Wisconsin-Madison)

Abstract

Self-rated health (SRH) is widely used to study health across a range of disciplines. However, relatively little research examines how features of its measurement in surveys influence respondents’ answers and the overall quality of the resulting measurement. Manipulations of response option order and scale orientation are particularly relevant to assess for SRH given the increasing prominence of web-based survey data collection and since these factors are often outside of the control of the researcher who is analyzing data collected by other investigators. We examine how the interplay of two features of SRH influence respondents’ answers in a 2-by-3 factorial experiment that varies (1) the order in which the response options are presented (“excellent” to “poor” or “poor” to “excellent”) and (2) the orientation of the response option scale (vertical, horizontal, or banked). The experiment was conducted online using workers from Amazon Mechanical Turk (N = 2945). We find no main effects of response scale orientation and no interaction between response option order and scale orientation. However, we find main effects of response option order: mean SRH and the proportion in “excellent” or “very good” health are higher (better) and the proportion in “fair” or “poor” health lower when the response options are ordered from “excellent” to “poor” compared to “poor” to “excellent.” We also see heterogeneous treatment effects of response option ordering across respondents’ characteristics associated with ability. Overall, the implications for the validity and cross-survey comparability of SRH are likely considerable for response option ordering and minimal for scale orientation.

Suggested Citation

  • Dana Garbarski & Nora Cate Schaeffer & Jennifer Dykema, 2019. "The Effects of Features of Survey Measurement on Self-Rated Health: Response Option Order and Scale Orientation," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 14(2), pages 545-560, April.
    • Handle: RePEc:spr:ariqol:v:14:y:2019:i:2:d:10.1007_s11482-018-9628-x
    DOI: 10.1007/s11482-018-9628-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11482-018-9628-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11482-018-9628-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2006. "Design of Web Questionnaires : The Effect of Layout in Rating Scales," Other publications TiSEM 9401f6c5-0275-400e-ac97-c, Tilburg University, School of Economics and Management.
    2. Vera Toepoel & Corrie Vis & Marcel Das & Arthur van Soest, 2009. "Design of Web Questionnaires," Sociological Methods & Research, , vol. 37(3), pages 371-392, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Giorgio Piccitto & Aart C. Liefbroer & Tom Emery, 2022. "Does the Survey Mode Affect the Association Between Subjective Well-being and its Determinants? An Experimental Comparison Between Face-to-Face and Web Mode," Journal of Happiness Studies, Springer, vol. 23(7), pages 3441-3461, October.
    2. Fabienne Wöhner & Axel Franzen, 2024. "The Impact of Answer Scale Orientation on the Measurement of Life Satisfaction," Journal of Happiness Studies, Springer, vol. 25(6), pages 1-18, August.
    3. Catherine P. Slade & Simon K. Medcalfe & C. Kevin Fortner & Kristin V. Walker, 2023. "Residential Segregation as a Policy Priority to Address Health Disparities: a Multilevel Analysis," Applied Research in Quality of Life, Springer;International Society for Quality-of-Life Studies, vol. 18(4), pages 1715-1735, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bart Buelens & Jan A. van den Brakel, 2015. "Measurement Error Calibration in Mixed-mode Sample Surveys," Sociological Methods & Research, , vol. 44(3), pages 391-426, August.
    2. Anna DeCastellarnau, 2018. "A classification of response scale characteristics that affect data quality: a literature review," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(4), pages 1523-1559, July.
    3. Brauner, Jacob, 2020. "Are Smileys Valid Answers? Survey Data Quality with Innovative Item Formats," SocArXiv dk9bc, Center for Open Science.
    4. Lindhjem, Henrik & Navrud, Ståle, 2011. "Using Internet in Stated Preference Surveys: A Review and Comparison of Survey Modes," International Review of Environmental and Resource Economics, now publishers, vol. 5(4), pages 309-351, September.
    5. Chatpong Tangmanee & Phattharaphong Niruttinanon, 2019. "Web Survey’s Completion Rates: Effects of Forced Responses, Question Display Styles, and Subjects’ Attitude," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 8(1), pages 20-29, January.
    6. Carina Cornesse & Annelies G. Blom, 2023. "Response Quality in Nonprobability and Probability-based Online Panels," Sociological Methods & Research, , vol. 52(2), pages 879-908, May.
    7. Fabo, B., 2017. "Towards an understanding of job matching using web data," Other publications TiSEM b8b877f2-ae6a-495f-b6cc-9, Tilburg University, School of Economics and Management.
    8. Weijters, Bert & Millet, Kobe & Cabooter, Elke, 2021. "Extremity in horizontal and vertical Likert scale format responses. Some evidence on how visual distance between response categories influences extreme responding," International Journal of Research in Marketing, Elsevier, vol. 38(1), pages 85-103.
    9. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Other publications TiSEM 21fbcc01-3c5d-4942-b41c-7, Tilburg University, School of Economics and Management.
    10. de Bruijne, M.A., 2015. "Designing web surveys for the multi-device internet," Other publications TiSEM 19e4d446-a62b-4a95-8691-8, Tilburg University, School of Economics and Management.
    11. Chatpong Tangmanee & Phattharaphong Niruttinanon, 2015. "Effects of Forced Responses and Question Display Styles on Web Survey Response Rates," International Journal of Research in Business and Social Science (2147-4478), Center for the Strategic Studies in Business and Finance, vol. 4(2), pages 54-62, April.
    12. Tobias Gummer & Tanja Kunz, 2022. "Relying on External Information Sources When Answering Knowledge Questions in Web Surveys," Sociological Methods & Research, , vol. 51(2), pages 816-836, May.
    13. Neuert Cornelia E. & Roßmann Joss & Silber Henning, 2023. "Using Eye-Tracking Methodology to Study Grid Question Designs in Web Surveys," Journal of Official Statistics, Sciendo, vol. 39(1), pages 79-101, March.
    14. Toepoel, V. & Das, J.W.M. & van Soest, A.H.O., 2008. "Design Effects in Web Surveys : Comparing Trained and Fresh Respondents," Discussion Paper 2008-51, Tilburg University, Center for Economic Research.
    15. Tobias Gummer & Joss Roßmann & Henning Silber, 2021. "Using Instructed Response Items as Attention Checks in Web Surveys: Properties and Implementation," Sociological Methods & Research, , vol. 50(1), pages 238-264, February.
    16. Toepoel, V. & Dillman, D.A., 2008. "Words, Numbers and Visual Heuristics in Web Surveys : Is there a Hierarchy of Importance?," Discussion Paper 2008-92, Tilburg University, Center for Economic Research.
    17. Yüksel, Atila, 2017. "A critique of “Response Bias” in the tourism, travel and hospitality research," Tourism Management, Elsevier, vol. 59(C), pages 376-384.
    18. Riccardo Testa & Giorgio Schifani & Giuseppina Migliore, 2021. "Understanding Consumers’ Convenience Orientation. An Exploratory Study of Fresh-Cut Fruit in Italy," Sustainability, MDPI, vol. 13(3), pages 1-13, January.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:ariqol:v:14:y:2019:i:2:d:10.1007_s11482-018-9628-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.