IDEAS home Printed from https://ideas.repec.org/a/spr/aodasc/v9y2022i2d10.1007_s40745-020-00253-5.html
   My bibliography  Save this article

A Comprehensive Survey of Loss Functions in Machine Learning

Author

Listed:
  • Qi Wang

    (University of Chinese Academy of Sciences
    Chinese Academy of Sciences
    Chinese Academy of Sciences)

  • Yue Ma

    (University of Chinese Academy of Sciences
    Chinese Academy of Sciences
    Chinese Academy of Sciences)

  • Kun Zhao

    (Beijing Wuzi University)

  • Yingjie Tian

    (Chinese Academy of Sciences
    Chinese Academy of Sciences
    University of Chinese Academy of Sciences)

Abstract

As one of the important research topics in machine learning, loss function plays an important role in the construction of machine learning algorithms and the improvement of their performance, which has been concerned and explored by many researchers. But it still has a big gap to summarize, analyze and compare the classical loss functions. Therefore, this paper summarizes and analyzes 31 classical loss functions in machine learning. Specifically, we describe the loss functions from the aspects of traditional machine learning and deep learning respectively. The former is divided into classification problem, regression problem and unsupervised learning according to the task type. The latter is subdivided according to the application scenario, and here we mainly select object detection and face recognition to introduces their loss functions. In each task or application, in addition to analyzing each loss function from formula, meaning, image and algorithm, the loss functions under the same task or application are also summarized and compared to deepen the understanding and provide help for the selection and improvement of loss function.

Suggested Citation

  • Qi Wang & Yue Ma & Kun Zhao & Yingjie Tian, 2022. "A Comprehensive Survey of Loss Functions in Machine Learning," Annals of Data Science, Springer, vol. 9(2), pages 187-212, April.
    • Handle: RePEc:spr:aodasc:v:9:y:2022:i:2:d:10.1007_s40745-020-00253-5
    DOI: 10.1007/s40745-020-00253-5
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s40745-020-00253-5
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s40745-020-00253-5?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Roger Koenker & Kevin F. Hallock, 2001. "Quantile Regression," Journal of Economic Perspectives, American Economic Association, vol. 15(4), pages 143-156, Fall.
    2. Lee, Yoonkyung & Lin, Yi & Wahba, Grace, 2004. "Multicategory Support Vector Machines: Theory and Application to the Classification of Microarray Data and Satellite Radiance Data," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 67-81, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Loutfi, Ahmad Amine & Sun, Mengtao & Loutfi, Ijlal & Solibakke, Per Bjarte, 2022. "Empirical study of day-ahead electricity spot-price forecasting: Insights into a novel loss function for training neural networks," Applied Energy, Elsevier, vol. 319(C).
    2. Emma King-Smith & Felix A. Faber & Usa Reilly & Anton V. Sinitskiy & Qingyi Yang & Bo Liu & Dennis Hyek & Alpha A. Lee, 2024. "Predictive Minisci late stage functionalization with transfer learning," Nature Communications, Nature, vol. 15(1), pages 1-13, December.
    3. Henrik Seckler & Ralf Metzler, 2022. "Bayesian deep learning for error estimation in the analysis of anomalous diffusion," Nature Communications, Nature, vol. 13(1), pages 1-13, December.
    4. Iqbal H. Sarker, 2023. "Machine Learning for Intelligent Data Analysis and Automation in Cybersecurity: Current and Future Prospects," Annals of Data Science, Springer, vol. 10(6), pages 1473-1498, December.
    5. Shuangyue Wang & Ziyan Luo, 2023. "Sparse Support Tensor Machine with Scaled Kernel Functions," Mathematics, MDPI, vol. 11(13), pages 1-20, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Akosah, Nana Kwame & Alagidede, Imhotep Paul & Schaling, Eric, 2020. "Testing for asymmetry in monetary policy rule for small-open developing economies: Multiscale Bayesian quantile evidence from Ghana," The Journal of Economic Asymmetries, Elsevier, vol. 22(C).
    2. Tabuga, Aubrey D., 2007. "International Remittances and Household Expenditures: the Philippine Case," Discussion Papers DP 2007-18, Philippine Institute for Development Studies.
    3. Paul Hewson & Keming Yu, 2008. "Quantile regression for binary performance indicators," Applied Stochastic Models in Business and Industry, John Wiley & Sons, vol. 24(5), pages 401-418, September.
    4. Alfred Michael Dockery & Sherry Bawa, 2015. "The Impact of Children on Australian Couples’ Wealth Accumulation," The Economic Record, The Economic Society of Australia, vol. 91(S1), pages 139-150, June.
    5. Héctor Manuel Zárate S., 2005. "Cambios en la estructura salarial: una historia desde la regresión cuanfílica," Monetaria, CEMLA, vol. 0(4), pages 339-364, octubre-d.
    6. Efobi, Uchenna & Asongu, Simplice & Okafor, Chinelo & Tchamyou, Vanessa & Tanankem, Belmondo, 2016. "Diaspora Remittance Inflow, Financial Development and the Industrialisation of Africa," MPRA Paper 76121, University Library of Munich, Germany.
    7. Corrado Andini, 2022. "Tertiary education for all and wage inequality: policy insights from quantile regression," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 50(6), pages 1281-1296, November.
    8. Fernando Antonio Slaibe Postali, 2016. "Oil windfalls and X-inefficiency: evidence from Brazil," Journal of Economic Studies, Emerald Group Publishing Limited, vol. 43(5), pages 699-718, October.
    9. Leon Zolotoy & Don O’Sullivan & Keke Song, 2021. "The Role of Ethical Standards in the Relationship Between Religious Social Norms and M&A Announcement Returns," Journal of Business Ethics, Springer, vol. 170(4), pages 721-742, May.
    10. Simplice A. Asongu & Nicholas M. Odhiambo, 2019. "Basic formal education quality, information technology, and inclusive human development in sub‐Saharan Africa," Sustainable Development, John Wiley & Sons, Ltd., vol. 27(3), pages 419-428, May.
    11. Aboura, Sofiane & Chevallier, Julien, 2016. "Spikes and crashes in the oil market," Research in International Business and Finance, Elsevier, vol. 36(C), pages 615-623.
    12. Chao, Shih-Kang & Härdle, Wolfgang K. & Yuan, Ming, 2021. "Factorisable Multitask Quantile Regression," Econometric Theory, Cambridge University Press, vol. 37(4), pages 794-816, August.
    13. Trojanek, Radoslaw & Huderek-Glapska, Sonia, 2018. "Measuring the noise cost of aviation – The association between the Limited Use Area around Warsaw Chopin Airport and property values," Journal of Air Transport Management, Elsevier, vol. 67(C), pages 103-114.
    14. Rodney Fort & Young Hoon Lee & Taeyeon Oh, 2019. "Quantile Insights on Market Structure and Worker Salaries: The Case of Major League Baseball," Journal of Sports Economics, , vol. 20(8), pages 1066-1087, December.
    15. Abramovich, Felix & Pensky, Marianna, 2019. "Classification with many classes: Challenges and pluses," Journal of Multivariate Analysis, Elsevier, vol. 174(C).
    16. Paulo M.M. Rodrigues & Rita Fradique Lourenço, 2015. "House prices: bubbles, exuberance or something else? Evidence from euro area countries," Working Papers w201517, Banco de Portugal, Economics and Research Department.
    17. repec:rre:publsh:v:39:y:2009:i:2:p:149-69 is not listed on IDEAS
    18. Muller, Christophe, 2018. "Heterogeneity and nonconstant effect in two-stage quantile regression," Econometrics and Statistics, Elsevier, vol. 8(C), pages 3-12.
    19. Juan Mora & Antonia Febrer, 2005. "Wage Distribution In Spain, 1994-1999: An Application Of A Flexible Estimator Of Conditional Distributions," Working Papers. Serie EC 2005-04, Instituto Valenciano de Investigaciones Económicas, S.A. (Ivie).
    20. Xiaoying Liu & Jere R. Behrman & Emily Hannum & Fan Wang & Qingguo Zhao, 2022. "Same environment, stratified impacts? Air pollution, extreme temperatures, and birth weight in south China," Papers 2204.00219, arXiv.org.
    21. Inklaar, Robert & Jong-A-Pin, Richard & de Haan, Jakob, 2008. "Trade and business cycle synchronization in OECD countries--A re-examination," European Economic Review, Elsevier, vol. 52(4), pages 646-666, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:aodasc:v:9:y:2022:i:2:d:10.1007_s40745-020-00253-5. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.