IDEAS home Printed from https://ideas.repec.org/a/bpj/jqsprt/v9y2013i2p187-202n7.html
   My bibliography  Save this article

Ranking rankings: an empirical comparison of the predictive power of sports ranking methods

Author

Listed:
  • Barrow Daniel

    (Pitzer College, Department of Mathematics, 1050 North Mills Avenue, Claremont, CA 91711, USA)

  • Drayer Ian

    (UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA)

  • Elliott Peter

    (UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA)

  • Gaut Garren

    (UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA)

  • Osting Braxton

    (UCLA, Department of Mathematics, 405 Hilgard Avenue, Los Angeles, CA 90095, USA)

Abstract

In this paper, we empirically evaluate the predictive power of eight sports ranking methods. For each ranking method, we implement two versions, one using only win-loss data and one utilizing score-differential data. The methods are compared on 4 datasets: 32 National Basketball Association (NBA) seasons, 112 Major League Baseball (MLB) seasons, 22 NCAA Division 1-A Basketball (NCAAB) seasons, and 56 NCAA Division 1-A Football (NCAAF) seasons. For each season of each dataset, we apply 20-fold cross validation to determine the predictive accuracy of the ranking methods. The non-parametric Friedman hypothesis test is used to assess whether the predictive errors for the considered rankings over the seasons are statistically dissimilar. The post-hoc Nemenyi test is then employed to determine which ranking methods have significant differences in predictive power. For all datasets, the null hypothesis – that all ranking methods are equivalent – is rejected at the 99% confidence level. For NCAAF and NCAAB datasets, the Nemenyi test concludes that the implementations utilizing score-differential data are usually more predictive than those using only win-loss data. For the NCAAF dataset, the least squares and random walker methods have significantly better predictive accuracy at the 95% confidence level than the other methods considered.

Suggested Citation

  • Barrow Daniel & Drayer Ian & Elliott Peter & Gaut Garren & Osting Braxton, 2013. "Ranking rankings: an empirical comparison of the predictive power of sports ranking methods," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 9(2), pages 187-202, June.
    • Handle: RePEc:bpj:jqsprt:v:9:y:2013:i:2:p:187-202:n:7
    DOI: 10.1515/jqas-2013-0013
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jqas-2013-0013
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/jqas-2013-0013?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Gill Ryan & Keating Jerome, 2009. "Assessing Methods for College Football Rankings," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 5(2), pages 1-21, May.
    2. Trono John A., 2010. "Rating/Ranking Systems, Post-Season Bowl Games, and "The Spread"," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 6(3), pages 1-20, July.
    3. Stefani Ray, 2011. "The Methodology of Officially Recognized International Sports Rating Systems," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 7(4), pages 1-22, October.
    4. Chan Victor, 2011. "Prediction Accuracy of Linear Models for Paired Comparisons in Sports," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 7(3), pages 1-35, July.
    5. Chartier Timothy P. & Kreutzer Erich & Langville Amy N & Pedings Kathryn E., 2011. "Sports Ranking with Nonuniform Weighting," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 7(3), pages 1-16, July.
    6. Burer Samuel, 2012. "Robust Rankings for College Football," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 8(2), pages 1-22, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Kovalchik, Stephanie, 2020. "Extension of the Elo rating system to margin of victory," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1329-1341.
    2. Fabian Wunderlich & Daniel Memmert, 2018. "The Betting Odds Rating System: Using soccer forecasts to forecast soccer," PLOS ONE, Public Library of Science, vol. 13(6), pages 1-18, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kovalchik Stephanie Ann, 2016. "Searching for the GOAT of tennis win prediction," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 12(3), pages 127-138, September.
    2. Subhasis Ray, 2021. "Identification of Research Paradigms for Managing the Cricketing Ecosystem Using Stakeholder Analysis and Text Mining," Management and Labour Studies, XLRI Jamshedpur, School of Business Management & Human Resources, vol. 46(3), pages 289-312, August.
    3. Karl Andrew T., 2012. "The Sensitivity of College Football Rankings to Several Modeling Choices," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 8(3), pages 1-44, October.
    4. Franceschet Massimo & Bozzo Enrico & Vidoni Paolo, 2017. "The temporalized Massey’s method," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 13(2), pages 37-48, June.
    5. Clive B Beggs & Alexander J Bond & Stacey Emmonds & Ben Jones, 2019. "Hidden dynamics of soccer leagues: The predictive ‘power’ of partial standings," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-28, December.
    6. Zhen, Jianzhe & den Hertog, Dick, 2016. "Centered Solutions for Uncertain Linear Equations (revision of CentER DP 2015-044)," Other publications TiSEM 297fa3b1-5290-48b5-bbc0-0, Tilburg University, School of Economics and Management.
    7. Lasek, Jan & Gagolewski, Marek, 2021. "Interpretable sports team rating models based on the gradient descent algorithm," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1061-1071.
    8. Devlin Stephen & Treloar Thomas & Creagar Molly & Cassels Samuel, 2021. "An iterative Markov rating method," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 17(2), pages 117-127, June.
    9. S. S. Dabadghao & B. Vaziri, 2022. "The predictive power of popular sports ranking methods in the NFL, NBA, and NHL," Operational Research, Springer, vol. 22(3), pages 2767-2783, July.
    10. B. Jay Coleman, 2014. "Minimum violations and predictive meta‐rankings for college football," Naval Research Logistics (NRL), John Wiley & Sons, vol. 61(1), pages 17-33, February.
    11. Wigness Maggie B & Williams Chadd C & Rowell Michael J, 2010. "A New Iterative Method for Ranking College Football Teams," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 6(2), pages 1-15, April.
    12. Kovalchik, Stephanie, 2020. "Extension of the Elo rating system to margin of victory," International Journal of Forecasting, Elsevier, vol. 36(4), pages 1329-1341.
    13. Zhen, Jianzhe & den Hertog, Dick, 2015. "Robust Solutions for Systems of Uncertain Linear Equations," Discussion Paper 2015-044, Tilburg University, Center for Economic Research.
    14. Zhen, Jianzhe & den Hertog, Dick, 2016. "Centered Solutions for Uncertain Linear Equations (revision of CentER DP 2015-044)," Discussion Paper 2016-048, Tilburg University, Center for Economic Research.
    15. Trono John A., 2010. "Rating/Ranking Systems, Post-Season Bowl Games, and "The Spread"," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 6(3), pages 1-20, July.
    16. Kovalchik, Stephanie & Reid, Machar, 2019. "A calibration method with dynamic updates for within-match forecasting of wins in tennis," International Journal of Forecasting, Elsevier, vol. 35(2), pages 756-766.
    17. Collingwood, James A.P. & Wright, Michael & Brooks, Roger J, 2022. "Evaluating the effectiveness of different player rating systems in predicting the results of professional snooker matches," European Journal of Operational Research, Elsevier, vol. 296(3), pages 1025-1035.
    18. Jianzhe Zhen & Dick Hertog, 2017. "Centered solutions for uncertain linear equations," Computational Management Science, Springer, vol. 14(4), pages 585-610, October.
    19. Stefani Ray, 2011. "The Methodology of Officially Recognized International Sports Rating Systems," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 7(4), pages 1-22, October.
    20. Zhen, Jianzhe & den Hertog, Dick, 2015. "Robust Solutions for Systems of Uncertain Linear Equations," Other publications TiSEM d072bdb9-4168-4522-90d8-1, Tilburg University, School of Economics and Management.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:jqsprt:v:9:y:2013:i:2:p:187-202:n:7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.