IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.16653.html
   My bibliography  Save this paper

The Credibility Transformer

Author

Listed:
  • Ronald Richman
  • Salvatore Scognamiglio
  • Mario V. Wuthrich

Abstract

Inspired by the large success of Transformers in Large Language Models, these architectures are increasingly applied to tabular data. This is achieved by embedding tabular data into low-dimensional Euclidean spaces resulting in similar structures as time-series data. We introduce a novel credibility mechanism to this Transformer architecture. This credibility mechanism is based on a special token that should be seen as an encoder that consists of a credibility weighted average of prior information and observation based information. We demonstrate that this novel credibility mechanism is very beneficial to stabilize training, and our Credibility Transformer leads to predictive models that are superior to state-of-the-art deep learning models.

Suggested Citation

  • Ronald Richman & Salvatore Scognamiglio & Mario V. Wuthrich, 2024. "The Credibility Transformer," Papers 2409.16653, arXiv.org.
    • Handle: RePEc:arx:papers:2409.16653
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2409.16653
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Gneiting, Tilmann, 2011. "Making and Evaluating Point Forecasts," Journal of the American Statistical Association, American Statistical Association, vol. 106(494), pages 746-762.
    2. Ronald Richman & Mario V. Wüthrich, 2023. "LocalGLMnet: interpretable deep learning for tabular data," Scandinavian Actuarial Journal, Taylor & Francis Journals, vol. 2023(1), pages 71-95, January.
    3. Kevin Kuo & Ronald Richman, 2021. "Embeddings and Attention in Predictive Modeling," Papers 2104.03545, arXiv.org.
    4. Gneiting, Tilmann & Raftery, Adrian E., 2007. "Strictly Proper Scoring Rules, Prediction, and Estimation," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 359-378, March.
    5. Delong, Łukasz & Kozak, Anna, 2023. "The use of autoencoders for training neural networks with mixed categorical and numerical features," ASTIN Bulletin, Cambridge University Press, vol. 53(2), pages 213-232, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Tobias Fissler & Yannick Hoga, 2024. "How to Compare Copula Forecasts?," Papers 2410.04165, arXiv.org.
    2. Rafael Frongillo, 2022. "Quantum Information Elicitation," Papers 2203.07469, arXiv.org.
    3. Lahiri, Kajal & Yang, Liu, 2013. "Forecasting Binary Outcomes," Handbook of Economic Forecasting, in: G. Elliott & C. Granger & A. Timmermann (ed.), Handbook of Economic Forecasting, edition 1, volume 2, chapter 0, pages 1025-1106, Elsevier.
    4. Tobias Fissler & Silvana M. Pesenti, 2022. "Sensitivity Measures Based on Scoring Functions," Papers 2203.00460, arXiv.org, revised Jul 2022.
    5. Knüppel, Malte & Schultefrankenfeld, Guido, 2019. "Assessing the uncertainty in central banks’ inflation outlooks," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1748-1769.
    6. Costa, Alexandre Bonnet R. & Ferreira, Pedro Cavalcanti G. & Gaglianone, Wagner P. & Guillén, Osmani Teixeira C. & Issler, João Victor & Lin, Yihao, 2021. "Machine learning and oil price point and density forecasting," Energy Economics, Elsevier, vol. 102(C).
    7. Anthony Coache & Sebastian Jaimungal, 2024. "Robust Reinforcement Learning with Dynamic Distortion Risk Measures," Papers 2409.10096, arXiv.org.
    8. Constandina Koki & Loukia Meligkotsidou & Ioannis Vrontos, 2020. "Forecasting under model uncertainty: Non‐homogeneous hidden Markov models with Pòlya‐Gamma data augmentation," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(4), pages 580-598, July.
    9. Emilio Zanetti Chini, 2018. "Forecasters’ utility and forecast coherence," CREATES Research Papers 2018-23, Department of Economics and Business Economics, Aarhus University.
    10. Weronika Nitka & Rafał Weron, 2023. "Combining predictive distributions of electricity prices. Does minimizing the CRPS lead to optimal decisions in day-ahead bidding?," Operations Research and Decisions, Wroclaw University of Science and Technology, Faculty of Management, vol. 33(3), pages 105-118.
    11. Sándor Baran & Patrícia Szokol & Marianna Szabó, 2021. "Truncated generalized extreme value distribution‐based ensemble model output statistics model for calibration of wind speed ensemble forecasts," Environmetrics, John Wiley & Sons, Ltd., vol. 32(6), September.
    12. Ravazzolo Francesco & Vahey Shaun P., 2014. "Forecast densities for economic aggregates from disaggregate ensembles," Studies in Nonlinear Dynamics & Econometrics, De Gruyter, vol. 18(4), pages 367-381, September.
    13. Yang, Dazhi & van der Meer, Dennis, 2021. "Post-processing in solar forecasting: Ten overarching thinking tools," Renewable and Sustainable Energy Reviews, Elsevier, vol. 140(C).
    14. Yeliz Ekinci & Nicoleta Serban & Ekrem Duman, 2021. "Optimal ATM replenishment policies under demand uncertainty," Operational Research, Springer, vol. 21(2), pages 999-1029, June.
    15. Knut Are Aastveit & Claudia Foroni & Francesco Ravazzolo, 2017. "Density Forecasts With Midas Models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 32(4), pages 783-801, June.
    16. Matteo Iacopini & Francesco Ravazzolo & Luca Rossini, 2020. "Proper scoring rules for evaluating asymmetry in density forecasting," Papers 2006.11265, arXiv.org, revised Sep 2020.
    17. Micha{l} Narajewski & Florian Ziel, 2021. "Optimal bidding in hourly and quarter-hourly electricity price auctions: trading large volumes of power with market impact and transaction costs," Papers 2104.14204, arXiv.org, revised Feb 2022.
    18. Tsyplakov, Alexander, 2013. "Evaluation of Probabilistic Forecasts: Proper Scoring Rules and Moments," MPRA Paper 45186, University Library of Munich, Germany.
    19. Fissler Tobias & Ziegel Johanna F., 2021. "On the elicitability of range value at risk," Statistics & Risk Modeling, De Gruyter, vol. 38(1-2), pages 25-46, January.
    20. Yuru Sun & Worapree Maneesoonthorn & Ruben Loaiza-Maya & Gael M. Martin, 2023. "Optimal probabilistic forecasts for risk management," Papers 2303.01651, arXiv.org.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.16653. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.