IDEAS home Printed from https://ideas.repec.org/a/gam/jrisks/v11y2023i6p110-d1166121.html
   My bibliography  Save this article

Investigating Causes of Model Instability: Properties of the Prediction Accuracy Index

Author

Listed:
  • Ross Taplin

    (Curtin Business School, Curtin University, Kent St, Bentley, WA 6102, Australia)

Abstract

The Prediction Accuracy Index (PAI) monitors stability, defined as whether the predictive power of a model has deteriorated due to a change in the distribution of the explanatory variables since its development. This paper shows how the PAI is related to the Mahalanobis distance, an established statistic for examining high leverage observations in data. This relationship is used to explore properties of the PAI, including tools for how the PAI can be decomposed into effects due to (a) individual observations/cases, (b) individual variables, and (c) shifts in the mean of variables. Not only are these tools useful for practitioners to help determine why models fail stability, but they also have implications for auditors and regulators. In particular, reasons why models containing econometric variables may fail stability are explored and suggestions to improve model development are discussed.

Suggested Citation

  • Ross Taplin, 2023. "Investigating Causes of Model Instability: Properties of the Prediction Accuracy Index," Risks, MDPI, vol. 11(6), pages 1-15, June.
    • Handle: RePEc:gam:jrisks:v:11:y:2023:i:6:p:110-:d:1166121
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-9091/11/6/110/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-9091/11/6/110/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Chamay Kruger & Willem Daniel Schutte & Tanja Verster, 2021. "Using Model Performance to Assess the Representativeness of Data for Model Development and Calibration in Financial Institutions," Risks, MDPI, vol. 9(11), pages 1-26, November.
    2. Mitchell A. Petersen, 2009. "Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches," The Review of Financial Studies, Society for Financial Studies, vol. 22(1), pages 435-480, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Leye Li & Louise Yi Lu & Dongyue Wang, 2022. "External labour market competitions and stock price crash risk: evidence from exposures to competitor CEOs’ award‐winning events," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(S1), pages 1421-1460, April.
    2. Kooyul Jung & Boyoung Kim & Byungmo Kim, 2009. "Tax Motivated Income Shifting and Korean Business Groups (Chaebol)," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 36(5‐6), pages 552-586, June.
    3. Greg Hebb, 2021. "On the performance of Bank-managed mutual funds: Canadian evidence," Journal of Economics and Finance, Springer;Academy of Economics and Finance, vol. 45(1), pages 22-48, January.
    4. Bradley Blaylock & Bradley P. Lawson & Michael A. Mayberry, 2020. "Taxable income, future profitability, and stock returns," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 47(7-8), pages 858-881, July.
    5. Boyer, M. Martin & Cowins, Elicia P. & Reddic, Willie D., 2019. "Portfolio rebalancing behavior with operating losses and investment regulation," International Review of Economics & Finance, Elsevier, vol. 63(C), pages 313-328.
    6. Claudia Buch & Catherine Koch & Michael Koetter, 2016. "Crises and rescues: liquidity transmission through international banks," BIS Working Papers 576, Bank for International Settlements.
    7. Caginalp, Gunduz & DeSantis, Mark, 2017. "Does price efficiency increase with trading volume? Evidence of nonlinearity and power laws in ETFs," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 467(C), pages 436-452.
    8. Gary F. Peters & Andrea M. Romi & Juan Manuel Sanchez, 2019. "The Influence of Corporate Sustainability Officers on Performance," Journal of Business Ethics, Springer, vol. 159(4), pages 1065-1087, November.
    9. Ji Seon Yoo & Ye Ji Lee, 2019. "National Culture and Tax Avoidance of Multinational Corporations," Sustainability, MDPI, vol. 11(24), pages 1-28, December.
    10. Martínez-Ferrero, Jennifer & García-Sánchez, Isabel-María, 2017. "Coercive, normative and mimetic isomorphism as determinants of the voluntary assurance of sustainability reports," International Business Review, Elsevier, vol. 26(1), pages 102-118.
    11. Liu, Xueli & Jiang, Chunxia & Wang, Feng & Yao, Shujie, 2021. "The impact of high-speed railway on urban housing prices in China: A network accessibility perspective," Transportation Research Part A: Policy and Practice, Elsevier, vol. 152(C), pages 84-99.
    12. Yue Zhao & Difang Wan, 2018. "Institutional high frequency trading and price discovery: Evidence from an emerging commodity futures market," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 38(2), pages 243-270, February.
    13. David Hirshleifer & Siew Hong Teoh & Jeff Jiewei Yu, 2011. "Short Arbitrage, Return Asymmetry, and the Accrual Anomaly," The Review of Financial Studies, Society for Financial Studies, vol. 24(7), pages 2429-2461.
    14. Stolowy, Hervé & Jeanjean, Thomas & Erkens, Michael, 2011. "The economic consequences of increasing the international visibility of financial reports," HEC Research Papers Series 957, HEC Paris.
    15. Chang, Sanders S. & Wang, F. Albert, 2015. "Adverse selection and the presence of informed trading," Journal of Empirical Finance, Elsevier, vol. 33(C), pages 19-33.
    16. Pavol Durana & Lucia Michalkova & Andrej Privara & Josef Marousek & Milos Tumpach, 2021. "Does the life cycle affect earnings management and bankruptcy?," Oeconomia Copernicana, Institute of Economic Research, vol. 12(2), pages 425-461, June.
    17. Chan-Jane Lin & Tawei Wang & Chao-Jung Pan, 2016. "Financial reporting quality and investment decisions for family firms," Asia Pacific Journal of Management, Springer, vol. 33(2), pages 499-532, June.
    18. Brunetti, M. & Ciciretti, R. & Djordjevic, Lj., 2016. "The determinants of household’s bank switching," Journal of Financial Stability, Elsevier, vol. 26(C), pages 175-189.
    19. Richardson, Grant & Taylor, Grantley & Lanis, Roman, 2015. "The impact of financial distress on corporate tax avoidance spanning the global financial crisis: Evidence from Australia," Economic Modelling, Elsevier, vol. 44(C), pages 44-53.
    20. Xin Qu & Majella Percy & Fang Hu & Jenny Stewart, 2022. "Can CEO equity‐based compensation limit investment‐related agency problems?," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2579-2614, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jrisks:v:11:y:2023:i:6:p:110-:d:1166121. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.