IDEAS home Printed from https://ideas.repec.org/a/gam/jrisks/v11y2023i6p110-d1166121.html
   My bibliography  Save this article

Investigating Causes of Model Instability: Properties of the Prediction Accuracy Index

Author

Listed:
  • Ross Taplin

    (Curtin Business School, Curtin University, Kent St, Bentley, WA 6102, Australia)

Abstract

The Prediction Accuracy Index (PAI) monitors stability, defined as whether the predictive power of a model has deteriorated due to a change in the distribution of the explanatory variables since its development. This paper shows how the PAI is related to the Mahalanobis distance, an established statistic for examining high leverage observations in data. This relationship is used to explore properties of the PAI, including tools for how the PAI can be decomposed into effects due to (a) individual observations/cases, (b) individual variables, and (c) shifts in the mean of variables. Not only are these tools useful for practitioners to help determine why models fail stability, but they also have implications for auditors and regulators. In particular, reasons why models containing econometric variables may fail stability are explored and suggestions to improve model development are discussed.

Suggested Citation

  • Ross Taplin, 2023. "Investigating Causes of Model Instability: Properties of the Prediction Accuracy Index," Risks, MDPI, vol. 11(6), pages 1-15, June.
    • Handle: RePEc:gam:jrisks:v:11:y:2023:i:6:p:110-:d:1166121
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-9091/11/6/110/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-9091/11/6/110/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Mitchell A. Petersen, 2009. "Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches," The Review of Financial Studies, Society for Financial Studies, vol. 22(1), pages 435-480, January.
    2. Chamay Kruger & Willem Daniel Schutte & Tanja Verster, 2021. "Using Model Performance to Assess the Representativeness of Data for Model Development and Calibration in Financial Institutions," Risks, MDPI, vol. 9(11), pages 1-26, November.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leye Li & Louise Yi Lu & Dongyue Wang, 2022. "External labour market competitions and stock price crash risk: evidence from exposures to competitor CEOs’ award‐winning events," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(S1), pages 1421-1460, April.
    2. Kooyul Jung & Boyoung Kim & Byungmo Kim, 2009. "Tax Motivated Income Shifting and Korean Business Groups (Chaebol)," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 36(5‐6), pages 552-586, June.
    3. Gary F. Peters & Andrea M. Romi & Juan Manuel Sanchez, 2019. "The Influence of Corporate Sustainability Officers on Performance," Journal of Business Ethics, Springer, vol. 159(4), pages 1065-1087, November.
    4. Liu, Xueli & Jiang, Chunxia & Wang, Feng & Yao, Shujie, 2021. "The impact of high-speed railway on urban housing prices in China: A network accessibility perspective," Transportation Research Part A: Policy and Practice, Elsevier, vol. 152(C), pages 84-99.
    5. David Hirshleifer & Siew Hong Teoh & Jeff Jiewei Yu, 2011. "Short Arbitrage, Return Asymmetry, and the Accrual Anomaly," The Review of Financial Studies, Society for Financial Studies, vol. 24(7), pages 2429-2461.
    6. Stolowy, Hervé & Jeanjean, Thomas & Erkens, Michael, 2011. "The economic consequences of increasing the international visibility of financial reports," HEC Research Papers Series 957, HEC Paris.
    7. Chang, Sanders S. & Wang, F. Albert, 2015. "Adverse selection and the presence of informed trading," Journal of Empirical Finance, Elsevier, vol. 33(C), pages 19-33.
    8. Pavol Durana & Lucia Michalkova & Andrej Privara & Josef Marousek & Milos Tumpach, 2021. "Does the life cycle affect earnings management and bankruptcy?," Oeconomia Copernicana, Institute of Economic Research, vol. 12(2), pages 425-461, June.
    9. Chan-Jane Lin & Tawei Wang & Chao-Jung Pan, 2016. "Financial reporting quality and investment decisions for family firms," Asia Pacific Journal of Management, Springer, vol. 33(2), pages 499-532, June.
    10. Xin Qu & Majella Percy & Fang Hu & Jenny Stewart, 2022. "Can CEO equity‐based compensation limit investment‐related agency problems?," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2579-2614, June.
    11. Lee, Seung Jung & Liu, Lucy Qian & Stebunovs, Viktors, 2022. "Risk-taking spillovers of U.S. monetary policy in the global market for U.S. dollar corporate loans," Journal of Banking & Finance, Elsevier, vol. 138(C).
    12. Zhu, Ling & Liu, Shasha & Kong, Dongmin, 2023. "Governments' fiscal stress and firm decentralization," International Review of Financial Analysis, Elsevier, vol. 90(C).
    13. Sebastian Eichfelder & Mona Lau, 2015. "Capitalization of capital gains taxes: (In)attention and turn-of-the-year returns," FEMM Working Papers 150019, Otto-von-Guericke University Magdeburg, Faculty of Economics and Management.
    14. Vink, Dennis & Nawas, Mike & van Breemen, Vivian, 2021. "Security design and credit rating risk in the CLO market," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 72(C).
    15. Andrea Orame, 2020. "The role of bank supply in the Italian credit market: evidence from a new regional survey," Temi di discussione (Economic working papers) 1279, Bank of Italy, Economic Research and International Relations Area.
    16. Xuanjuan Chen & Jing-Zhi Huang & Zhenzhen Sun & Tong Yao & Tong Yu, 2020. "Liquidity Premium in the Eye of the Beholder: An Analysis of the Clientele Effect in the Corporate Bond Market," Management Science, INFORMS, vol. 66(2), pages 932-957, February.
    17. Chan, Kalok & Yang, Jian & Zhou, Yinggang, 2018. "Conditional co-skewness and safe-haven currencies: A regime switching approach," Journal of Empirical Finance, Elsevier, vol. 48(C), pages 58-80.
    18. Amrit Amirapu, 2021. "Justice Delayed Is Growth Denied: The Effect of Slow Courts on Relationship-Specific Industries in India," Economic Development and Cultural Change, University of Chicago Press, vol. 70(1), pages 415-451.
    19. Chen, Shenglan & Ma, Hui & Teng, Haimeng & Wu, Qiang, 2022. "Banking liberalization and corporate tax planning: Evidence from natural experiments," Journal of Corporate Finance, Elsevier, vol. 76(C).
    20. Ghafran, Chaudhry & O'Sullivan, Noel, 2017. "The impact of audit committee expertise on audit quality: Evidence from UK audit fees," The British Accounting Review, Elsevier, vol. 49(6), pages 578-593.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jrisks:v:11:y:2023:i:6:p:110-:d:1166121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.