IDEAS home Printed from https://ideas.repec.org/a/spr/psycho/v84y2019i1d10.1007_s11336-018-9648-3.html
   My bibliography  Save this article

A Simple Method for Comparing Complex Models: Bayesian Model Comparison for Hierarchical Multinomial Processing Tree Models Using Warp-III Bridge Sampling

Author

Listed:
  • Quentin F. Gronau

    (University of Amsterdam)

  • Eric-Jan Wagenmakers

    (University of Amsterdam)

  • Daniel W. Heck

    (University of Mannheim)

  • Dora Matzke

    (University of Amsterdam)

Abstract

Multinomial processing trees (MPTs) are a popular class of cognitive models for categorical data. Typically, researchers compare several MPTs, each equipped with many parameters, especially when the models are implemented in a hierarchical framework. A Bayesian solution is to compute posterior model probabilities and Bayes factors. Both quantities, however, rely on the marginal likelihood, a high-dimensional integral that cannot be evaluated analytically. In this case study, we show how Warp-III bridge sampling can be used to compute the marginal likelihood for hierarchical MPTs. We illustrate the procedure with two published data sets and demonstrate how Warp-III facilitates Bayesian model averaging.

Suggested Citation

  • Quentin F. Gronau & Eric-Jan Wagenmakers & Daniel W. Heck & Dora Matzke, 2019. "A Simple Method for Comparing Complex Models: Bayesian Model Comparison for Hierarchical Multinomial Processing Tree Models Using Warp-III Bridge Sampling," Psychometrika, Springer;The Psychometric Society, vol. 84(1), pages 261-284, March.
    • Handle: RePEc:spr:psycho:v:84:y:2019:i:1:d:10.1007_s11336-018-9648-3
    DOI: 10.1007/s11336-018-9648-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11336-018-9648-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11336-018-9648-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Dora Matzke & Conor Dolan & William Batchelder & Eric-Jan Wagenmakers, 2015. "Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 205-235, March.
    2. Overstall, Antony M. & Forster, Jonathan J., 2010. "Default Bayesian model determination methods for generalised linear mixed models," Computational Statistics & Data Analysis, Elsevier, vol. 54(12), pages 3269-3288, December.
    3. De Boeck, Paul & Partchev, Ivailo, 2012. "IRTrees: Tree-Based Item Response Models of the GLMM Family," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(c01).
    4. Maydeu-Olivares, Albert & Joe, Harry, 2005. "Limited- and Full-Information Estimation and Goodness-of-Fit Testing in 2n Contingency Tables: A Unified Framework," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 1009-1020, September.
    5. David J. Spiegelhalter & Nicola G. Best & Bradley P. Carlin & Angelika Van Der Linde, 2002. "Bayesian measures of model complexity and fit," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(4), pages 583-639, October.
    6. Karl Klauer, 2010. "Hierarchical Multinomial Processing Tree Models: A Latent-Trait Approach," Psychometrika, Springer;The Psychometric Society, vol. 75(1), pages 70-98, March.
    7. Xiangen Hu & William Batchelder, 1994. "The statistical analysis of general processing tree models with the EM algorithm," Psychometrika, Springer;The Psychometric Society, vol. 59(1), pages 21-47, March.
    8. Daniel W. Heck & Edgar Erdfelder & Pascal J. Kieslich, 2018. "Generalized Processing Tree Models: Jointly Modeling Discrete and Continuous Variables," Psychometrika, Springer;The Psychometric Society, vol. 83(4), pages 893-918, December.
    9. Karl Klauer, 2006. "Hierarchical Multinomial Processing Tree Models: A Latent-Class Approach," Psychometrika, Springer;The Psychometric Society, vol. 71(1), pages 7-31, March.
    10. Eddelbuettel, Dirk & Francois, Romain, 2011. "Rcpp: Seamless R and C++ Integration," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 40(i08).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gronau, Quentin F. & Bennett, Murray S. & Brown, Scott D. & Hawkins, Guy E. & Eidels, Ami, 2023. "Do choice tasks and rating scales elicit the same judgments?," Journal of choice modelling, Elsevier, vol. 49(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Steffen Nestler & Edgar Erdfelder, 2023. "Random Effects Multinomial Processing Tree Models: A Maximum Likelihood Approach," Psychometrika, Springer;The Psychometric Society, vol. 88(3), pages 809-829, September.
    2. Dora Matzke & Conor Dolan & William Batchelder & Eric-Jan Wagenmakers, 2015. "Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items," Psychometrika, Springer;The Psychometric Society, vol. 80(1), pages 205-235, March.
    3. Florian Wickelmaier & Achim Zeileis, 2016. "Using Recursive Partitioning to Account for Parameter Heterogeneity in Multinomial Processing Tree Models," Working Papers 2016-26, Faculty of Economics and Statistics, Universität Innsbruck.
    4. Sun-Joo Cho & Sarah Brown-Schmidt & Paul De Boeck & Jianhong Shen, 2020. "Modeling Intensive Polytomous Time-Series Eye-Tracking Data: A Dynamic Tree-Based Item Response Model," Psychometrika, Springer;The Psychometric Society, vol. 85(1), pages 154-184, March.
    5. Daniel W. Heck & Edgar Erdfelder & Pascal J. Kieslich, 2018. "Generalized Processing Tree Models: Jointly Modeling Discrete and Continuous Variables," Psychometrika, Springer;The Psychometric Society, vol. 83(4), pages 893-918, December.
    6. Laine Bradshaw & Jonathan Templin, 2014. "Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions," Psychometrika, Springer;The Psychometric Society, vol. 79(3), pages 403-425, July.
    7. Buddhavarapu, Prasad & Scott, James G. & Prozzi, Jorge A., 2016. "Modeling unobserved heterogeneity using finite mixture random parameters for spatially correlated discrete count data," Transportation Research Part B: Methodological, Elsevier, vol. 91(C), pages 492-510.
    8. Marta Castela & Edgar Erdfelder, 2017. "Further evidence for the memory state heuristic: Recognition latency predictions for binary inferences," Judgment and Decision Making, Society for Judgment and Decision Making, vol. 12(6), pages 537-552, November.
    9. Dylan Molenaar & Paul Boeck, 2018. "Response Mixture Modeling: Accounting for Heterogeneity in Item Characteristics across Response Times," Psychometrika, Springer;The Psychometric Society, vol. 83(2), pages 279-297, June.
    10. Zita Oravecz & Royce Anders & William Batchelder, 2015. "Hierarchical Bayesian Modeling for Test Theory Without an Answer Key," Psychometrika, Springer;The Psychometric Society, vol. 80(2), pages 341-364, June.
    11. Mulder, Joris, 2014. "Prior adjusted default Bayes factors for testing (in)equality constrained hypotheses," Computational Statistics & Data Analysis, Elsevier, vol. 71(C), pages 448-463.
    12. repec:cup:judgdm:v:12:y:2017:i:6:p:537-552 is not listed on IDEAS
    13. Carter Allen & Yuzhou Chang & Brian Neelon & Won Chang & Hang J. Kim & Zihai Li & Qin Ma & Dongjun Chung, 2023. "A Bayesian multivariate mixture model for high throughput spatial transcriptomics," Biometrics, The International Biometric Society, vol. 79(3), pages 1775-1787, September.
    14. Minjeong Jeon & Paul Boeck & Jevan Luo & Xiangrui Li & Zhong-Lin Lu, 2021. "Modeling Within-Item Dependencies in Parallel Data on Test Responses and Brain Activation," Psychometrika, Springer;The Psychometric Society, vol. 86(1), pages 239-271, March.
    15. Benjamin W. Domingue & Klint Kanopka & Radhika Kapoor & Steffi Pohl & R. Philip Chalmers & Charles Rahal & Mijke Rhemtulla, 2024. "The InterModel Vigorish as a Lens for Understanding (and Quantifying) the Value of Item Response Models for Dichotomously Coded Items," Psychometrika, Springer;The Psychometric Society, vol. 89(3), pages 1034-1054, September.
    16. George Gerogiannis & Mark Tranmer & Duncan Lee & Thomas Valente, 2022. "A Bayesian spatio‐network model for multiple adolescent adverse health behaviours," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(2), pages 271-287, March.
    17. Steven Andrew Culpepper & Yinghan Chen, 2019. "Development and Application of an Exploratory Reduced Reparameterized Unified Model," Journal of Educational and Behavioral Statistics, , vol. 44(1), pages 3-24, February.
    18. Erick da Conceição Amorim & Vinícius Diniz Mayrink, 2020. "Clustering non-linear interactions in factor analysis," METRON, Springer;Sapienza Università di Roma, vol. 78(3), pages 329-352, December.
    19. Buddhavarapu, Prasad & Bansal, Prateek & Prozzi, Jorge A., 2021. "A new spatial count data model with time-varying parameters," Transportation Research Part B: Methodological, Elsevier, vol. 150(C), pages 566-586.
    20. Mumtaz, Haroon & Theodoridis, Konstantinos, 2017. "Common and country specific economic uncertainty," Journal of International Economics, Elsevier, vol. 105(C), pages 205-216.
    21. Jesse Elliott & Zemin Bai & Shu-Ching Hsieh & Shannon E Kelly & Li Chen & Becky Skidmore & Said Yousef & Carine Zheng & David J Stewart & George A Wells, 2020. "ALK inhibitors for non-small cell lung cancer: A systematic review and network meta-analysis," PLOS ONE, Public Library of Science, vol. 15(2), pages 1-18, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:psycho:v:84:y:2019:i:1:d:10.1007_s11336-018-9648-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.