IDEAS home Printed from https://ideas.repec.org/a/sae/jedbes/v41y2016i3p239-268.html
   My bibliography  Save this article

Response Styles in Rating Scales

Author

Listed:
  • Gerhard Tutz
  • Moritz Berger

    (Ludwig-Maximilians–Universität München)

Abstract

Heterogeneity in response styles can affect the conclusions drawn from rating scale data. In particular, biased estimates can be expected if one ignores a tendency to middle categories or to extreme categories. An adjacent categories model is proposed that simultaneously models the content-related effects and the heterogeneity in response styles. By accounting for response styles, it provides a simple remedy for the bias that occurs if the response style is ignored. The model allows to include explanatory variables that have a content-related effect as well as an effect on the response style. A visualization tool is developed that makes the interpretation of effects easily accessible. The proposed model is embedded into the framework of multivariate generalized linear model, which entails that common estimation and inference tools can be used. Existing software can be used to fit the model, which makes it easy to apply.

Suggested Citation

  • Gerhard Tutz & Moritz Berger, 2016. "Response Styles in Rating Scales," Journal of Educational and Behavioral Statistics, , vol. 41(3), pages 239-268, June.
    • Handle: RePEc:sae:jedbes:v:41:y:2016:i:3:p:239-268
    DOI: 10.3102/1076998616636850
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.3102/1076998616636850
    Download Restriction: no
    File URL: https://libkey.io/10.3102/1076998616636850?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Weijters, Bert & Cabooter, Elke & Schillewaert, Niels, 2010. "The effect of rating scale format on response styles: The number of response categories and response category labels," International Journal of Research in Marketing, Elsevier, vol. 27(3), pages 236-247.
    2. Geoff Masters, 1982. "A rasch model for partial credit scoring," Psychometrika, Springer;The Psychometric Society, vol. 47(2), pages 149-174, June.
    3. Guy Moors, 2003. "Diagnosing Response Style Behavior by Means of a Latent-Class Factor Approach. Socio-Demographic Correlates of Gender Role Attitudes and Perceptions of Ethnic Discrimination Reexamined," Quality & Quantity: International Journal of Methodology, Springer, vol. 37(3), pages 277-302, August.
    4. De Boeck, Paul & Partchev, Ivailo, 2012. "IRTrees: Tree-Based Item Response Models of the GLMM Family," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(c01).
    5. Youngsuk Suh & Daniel Bolt, 2010. "Nested Logit Models for Multiple-Choice Item Response Data," Psychometrika, Springer;The Psychometric Society, vol. 75(3), pages 454-473, September.
    6. Romina Gambacorta & Maria Iannario, 2013. "Measuring Job Satisfaction with CUB Models," LABOUR, CEIS, vol. 27(2), pages 198-224, June.
    7. R. Darrell Bock, 1972. "Estimating item parameters and latent ability when responses are scored in two or more nominal categories," Psychometrika, Springer;The Psychometric Society, vol. 37(1), pages 29-51, March.
    8. van Herk, H. & Poortinga, Y.H. & Verhallen, T.M.M., 2004. "Response styles in rating scales : Evidence of method bias in data from 6 EU countries," Other publications TiSEM c8befc7a-f2f4-44cf-b2fc-b, Tilburg University, School of Economics and Management.
    9. Timothy Johnson, 2003. "On the use of heterogeneous thresholds ordinal regression models to account for individual differences in response style," Psychometrika, Springer;The Psychometric Society, vol. 68(4), pages 563-583, December.
    10. Yee, Thomas W., 2010. "The VGAM Package for Categorical Data Analysis," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 32(i10).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Anne Thissen-Roe & David Thissen, 2013. "A Two-Decision Model for Responses to Likert-Type Items," Journal of Educational and Behavioral Statistics, , vol. 38(5), pages 522-547, October.
    2. Timothy R. Johnson & Daniel M. Bolt, 2010. "On the Use of Factor-Analytic Multinomial Logit Item Response Models to Account for Individual Differences in Response Style," Journal of Educational and Behavioral Statistics, , vol. 35(1), pages 92-114, February.
    3. Gerhard Tutz, 2021. "Hierarchical Models for the Analysis of Likert Scales in Regression and Item Response Analysis," International Statistical Review, International Statistical Institute, vol. 89(1), pages 18-35, April.
    4. Luo, Nanyu & Ji, Feng & Han, Yuting & He, Jinbo & Zhang, Xiaoya, 2024. "Fitting item response theory models using deep learning computational frameworks," OSF Preprints tjxab, Center for Open Science.
    5. Ulf Böckenholt, 2012. "The Cognitive-Miser Response Model: Testing for Intuitive and Deliberate Reasoning," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 388-399, April.
    6. Kuan-Yu Jin & Yi-Jhen Wu & Hui-Fang Chen, 2022. "A New Multiprocess IRT Model With Ideal Points for Likert-Type Items," Journal of Educational and Behavioral Statistics, , vol. 47(3), pages 297-321, June.
    7. Albert Maydeu-Olivares & Harry Joe, 2006. "Limited Information Goodness-of-fit Testing in Multidimensional Contingency Tables," Psychometrika, Springer;The Psychometric Society, vol. 71(4), pages 713-732, December.
    8. Brooke E. Magnus & David Thissen, 2017. "Item Response Modeling of Multivariate Count Data With Zero Inflation, Maximum Inflation, and Heaping," Journal of Educational and Behavioral Statistics, , vol. 42(5), pages 531-558, October.
    9. Yingbin Zhang & Zhaoxi Yang & Yehui Wang, 2022. "The Impact of Extreme Response Style on the Mean Comparison of Two Independent Samples," SAGE Open, , vol. 12(2), pages 21582440221, June.
    10. repec:jss:jstsof:35:i12 is not listed on IDEAS
    11. Maydeu Olivares, Alberto & D'Zurilla, Thomas J. & Morera, Osvaldo, 1996. "Assessing measurement invariance in questionnaires within latent trait models using item response theory," DES - Working Papers. Statistics and Econometrics. WS 10456, Universidad Carlos III de Madrid. Departamento de Estadística.
    12. Daniel Bolt & James Wollack & Youngsuk Suh, 2012. "Application of a Multidimensional Nested Logit Model to Multiple-Choice Test Items," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 339-357, April.
    13. Dylan Molenaar, 2015. "Heteroscedastic Latent Trait Models for Dichotomous Data," Psychometrika, Springer;The Psychometric Society, vol. 80(3), pages 625-644, September.
    14. Piotr Białowolski, 2016. "The influence of negative response style on survey-based household inflation expectations," Quality & Quantity: International Journal of Methodology, Springer, vol. 50(2), pages 509-528, March.
    15. De Boeck, Paul & Partchev, Ivailo, 2012. "IRTrees: Tree-Based Item Response Models of the GLMM Family," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(c01).
    16. Geofferey Masters, 1985. "A comparison of latent trait and latent class analyses of Likert-type data," Psychometrika, Springer;The Psychometric Society, vol. 50(1), pages 69-82, March.
    17. Daphna Harel & Russell J. Steele, 2018. "An Information Matrix Test for the Collapsing of Categories Under the Partial Credit Model," Journal of Educational and Behavioral Statistics, , vol. 43(6), pages 721-750, December.
    18. Singh, Jagdip, 2004. "Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example," Journal of Business Research, Elsevier, vol. 57(2), pages 184-208, February.
    19. David Magis, 2015. "A Note on the Equivalence Between Observed and Expected Information Functions With Polytomous IRT Models," Journal of Educational and Behavioral Statistics, , vol. 40(1), pages 96-105, February.
    20. Tellis, Gerard J. & Chandrasekaran, Deepa, 2010. "Extent and impact of response biases in cross-national survey research," International Journal of Research in Marketing, Elsevier, vol. 27(4), pages 329-341.
    21. Myszkowski, Nils & Storme, Martin, 2018. "A snapshot of g? Binary and polytomous item-response theory investigations of the last series of the Standard Progressive Matrices (SPM-LS)," Intelligence, Elsevier, vol. 68(C), pages 109-116.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:jedbes:v:41:y:2016:i:3:p:239-268. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.