IDEAS home Printed from https://ideas.repec.org/a/bla/jorssc/v71y2022i1p91-118.html
   My bibliography  Save this article

Urnings: A new method for tracking dynamically changing parameters in paired comparison systems

Author

Listed:
  • Maria Bolsinova
  • Gunter Maris
  • Abe D. Hofman
  • Han L. J. van der Maas
  • Matthieu J. S. Brinkhuis

Abstract

We introduce a new rating system for tracking the development of parameters based on a stream of observations that can be viewed as paired comparisons. Rating systems are applied in competitive games, adaptive learning systems and platforms for product and service reviews. We model each observation as an outcome of a game of chance that depends on the parameters of interest (e.g. the outcome of a chess game depends on the abilities of the two players). Determining the probabilities of the different game outcomes is conceptualized as an urn problem, where a rating is represented by a probability (i.e. proportion of balls in the urn). This setup allows for evaluating the standard errors of the ratings and performing statistical inferences about the development of, and relations between, parameters. Theoretical properties of the system in terms of the invariant distributions of the ratings and their convergence are derived. The properties of the rating system are illustrated with simulated examples and its potential for answering research questions is illustrated using data from competitive chess, a movie review system, and an adaptive learning system for math.

Suggested Citation

  • Maria Bolsinova & Gunter Maris & Abe D. Hofman & Han L. J. van der Maas & Matthieu J. S. Brinkhuis, 2022. "Urnings: A new method for tracking dynamically changing parameters in paired comparison systems," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(1), pages 91-118, January.
    • Handle: RePEc:bla:jorssc:v:71:y:2022:i:1:p:91-118
    DOI: 10.1111/rssc.12523
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssc.12523
    Download Restriction: no
    File URL: https://libkey.io/10.1111/rssc.12523?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Lehmann, Robert & Wohlrabe, Klaus, 2017. "Who is the ‘Journal Grand Master’? A new ranking based on the Elo rating system," Journal of Informetrics, Elsevier, vol. 11(3), pages 800-809.
    2. Mark Glickman, 2001. "Dynamic paired comparison models with stochastic variances," Journal of Applied Statistics, Taylor & Francis Journals, vol. 28(6), pages 673-689.
    3. Hvattum, Lars Magnus & Arntzen, Halvard, 2010. "Using ELO ratings for match result prediction in association football," International Journal of Forecasting, Elsevier, vol. 26(3), pages 460-470, July.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Singh, Aaditya & Scarf, Phil & Baker, Rose, 2023. "A unified theory for bivariate scores in possessive ball-sports: The case of handball," European Journal of Operational Research, Elsevier, vol. 304(3), pages 1099-1112.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hua, Hsuan-Fu & Chang, Ching-Ju & Lin, Tse-Ching & Weng, Ruby Chiu-Hsing, 2024. "Rating players by Laplace’s approximation and dynamic modeling," International Journal of Forecasting, Elsevier, vol. 40(3), pages 1152-1165.
    2. Yamini Nekkanti & Dibyojyoti Bhattacharjee, 2020. "Novel Performance Metrics to Evaluate the Duel Between a Batsman and a Bowler," Management and Labour Studies, XLRI Jamshedpur, School of Business Management & Human Resources, vol. 45(2), pages 201-211, May.
    3. Kovalchik, Stephanie, 2020. "Extension of the Elo rating system to margin of victory," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1329-1341.
    4. Paul Bose & Eberhard Feess & Helge Mueller, 2022. "Favoritism towards High-Status Clubs: Evidence from German Soccer," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 38(2), pages 422-478.
    5. Bruzzone, Octavio A. & Logarzo, Guillermo A. & Aguirre, María B. & Virla, Eduardo G., 2018. "Intra-host interspecific larval parasitoid competition solved using modelling and bayesian statistics," Ecological Modelling, Elsevier, vol. 385(C), pages 114-123.
    6. Chowdhury, Subhasish M. & Jewell, Sarah & Singleton, Carl, 2024. "Can awareness reduce (and reverse) identity-driven bias in judgement? Evidence from international cricket," Journal of Economic Behavior & Organization, Elsevier, vol. 226(C).
    7. Csató, László, 2023. "How to avoid uncompetitive games? The importance of tie-breaking rules," European Journal of Operational Research, Elsevier, vol. 307(3), pages 1260-1269.
    8. Baboota, Rahul & Kaur, Harleen, 2019. "Predictive analysis and modelling football results using machine learning approach for English Premier League," International Journal of Forecasting, Elsevier, vol. 35(2), pages 741-755.
    9. Angelini, Giovanni & Candila, Vincenzo & De Angelis, Luca, 2022. "Weighted Elo rating for tennis match predictions," European Journal of Operational Research, Elsevier, vol. 297(1), pages 120-132.
    10. Csató, László & Bodnár, Gergely, 2023. "Mérhetnénk jobban a csapatok erejét a Bajnokok Ligájában? Fontos megjegyzés az Európai Labdarúgó-szövetség számára [How to better measure team strength in the Champions League. An important message," Közgazdasági Szemle (Economic Review - monthly of the Hungarian Academy of Sciences), Közgazdasági Szemle Alapítvány (Economic Review Foundation), vol. 0(7), pages 813-827.
    11. Baker, Rose D. & McHale, Ian G., 2017. "An empirical Bayes model for time-varying paired comparisons ratings: Who is the greatest women’s tennis player?," European Journal of Operational Research, Elsevier, vol. 258(1), pages 328-333.
    12. Alexandra Grand & Regina Dittrich & Brian Francis, 2015. "Markov models of dependence in longitudinal paired comparisons: an application to course design," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 99(2), pages 237-257, April.
    13. Blaž Krese & Erik Štrumbelj, 2021. "A Bayesian approach to time-varying latent strengths in pairwise comparisons," PLOS ONE, Public Library of Science, vol. 16(5), pages 1-17, May.
    14. J. James Reade & Jan C. van Ours, 2024. "Consumer perceptions matter: A case study of an anomaly in English football," Tinbergen Institute Discussion Papers 24-023/V, Tinbergen Institute.
    15. Sarah Jewell & J. James Reade & Carl Singleton, 2020. "It's Just Not Cricket: The Uncontested Toss and the Gentleman's Game," Economics Discussion Papers em-dp2020-10, Department of Economics, University of Reading.
    16. He, Xue-Zhong & Treich, Nicolas, 2017. "Prediction market prices under risk aversion and heterogeneous beliefs," Journal of Mathematical Economics, Elsevier, vol. 70(C), pages 105-114.
    17. Roberto Gásquez & Vicente Royuela, 2016. "The Determinants of International Football Success: A Panel Data Analysis of the Elo Rating," Social Science Quarterly, Southwestern Social Science Association, vol. 97(2), pages 125-141, June.
    18. da Costa, Igor Barbosa & Marinho, Leandro Balby & Pires, Carlos Eduardo Santos, 2022. "Forecasting football results and exploiting betting markets: The case of “both teams to score”," International Journal of Forecasting, Elsevier, vol. 38(3), pages 895-909.
    19. J. James Reade & Dominik Schreyer & Carl Singleton, 2022. "Eliminating supportive crowds reduces referee bias," Economic Inquiry, Western Economic Association International, vol. 60(3), pages 1416-1436, July.
    20. L.F.M. Groot & J. Ferwerda, 2014. "Soccer jersey sponsors and the world cup," Working Papers 14-07, Utrecht School of Economics.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssc:v:71:y:2022:i:1:p:91-118. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.