IDEAS home Printed from https://ideas.repec.org/a/wly/navres/v58y2011i3p236-254.html
   My bibliography  Save this article

A confidence paradigm for classification systems

Author

Listed:
  • Nathan J. Leap
  • Kenneth W. Bauer

Abstract

There is no universally accepted methodology to determine how much confidence one should place in the output of a classification system. In this article, we develop a confidence paradigm. This is a theoretical framework that attempts to unite the viewpoints of the classification system developer (or engineer) and the classification system user (or warfighter). The developer designs and tests the classification system at a macro‐level. The user fields the system in an environment often quite different than the environment used to develop the system. The user operates at a micro‐level and is interested in the indications as they are made by the system. The paradigm is based on the assumptions that the system confidence acts like or can be modelled as value, and that indication confidence can be modelled as a function of the posterior probability estimates. The viewpoints of the developer and the user are unified through the fundamental proposition that the expected value of the user's confidence should be approximately equal to the developer's confidence. This paradigm provides a direct link between traditional decision analysis techniques and traditional pattern recognition techniques. This methodology is applied to an automatic target recognition data set, and the results demonstrate the sort of behavior that would be expected from a rational confidence measure. © 2011 Wiley Periodicals, Inc. Naval Research Logistics, 2011

Suggested Citation

  • Nathan J. Leap & Kenneth W. Bauer, 2011. "A confidence paradigm for classification systems," Naval Research Logistics (NRL), John Wiley & Sons, vol. 58(3), pages 236-254, April.
    • Handle: RePEc:wly:navres:v:58:y:2011:i:3:p:236-254
    DOI: 10.1002/nav.20426
    as

    Download full text from publisher

    File URL: https://doi.org/10.1002/nav.20426
    Download Restriction: no
    File URL: https://libkey.io/10.1002/nav.20426?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. James S. Dyer & Rakesh K. Sarin, 1979. "Measurable Multiattribute Value Functions," Operations Research, INFORMS, vol. 27(4), pages 810-822, August.
    2. Wojtek J. Krzanowski & Trevor C. Bailey & Derek Partridge & Jonathan E. Fieldsend & Richard M. Everson & Vitaly Schetinin, 2006. "Confidence in Classification: A Bayesian Approach," Journal of Classification, Springer;The Classification Society, vol. 23(2), pages 199-220, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Janne Gustafsson, 2020. "Valuation of Research and Development Projects Using Buying and Selling Prices: Generalized Definitions," Decision Analysis, INFORMS, vol. 17(2), pages 154-168, June.
    2. Peter Reichert & Klemens Niederberger & Peter Rey & Urs Helg & Susanne Haertel-Borer, 2019. "The need for unconventional value aggregation techniques: experiences from eliciting stakeholder preferences in environmental management," EURO Journal on Decision Processes, Springer;EURO - The Association of European Operational Research Societies, vol. 7(3), pages 197-219, November.
    3. Wynn C. Stirling & Teppo Felin, 2016. "Satisficing, preferences, and social interaction: a new perspective," Theory and Decision, Springer, vol. 81(2), pages 279-308, August.
    4. Christophe Labreuche & Michel Grabisch, 2016. "A comparison of the GAI model and the Choquet integral with respect to a k-ary capacity," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01277825, HAL.
    5. Jay Simon, 2016. "On the existence of altruistic value and utility functions," Theory and Decision, Springer, vol. 81(3), pages 371-391, September.
    6. James E. Smith & James S. Dyer, 2021. "On (Measurable) Multiattribute Value Functions: An Expository Argument," Decision Analysis, INFORMS, vol. 18(4), pages 247-256, December.
    7. Marttunen, Mika & Haara, Arto & Hjerppe, Turo & Kurttila, Mikko & Liesiö, Juuso & Mustajoki, Jyri & Saarikoski, Heli & Tolvanen, Anne, 2023. "Parallel and comparative use of three multicriteria decision support methods in an environmental portfolio problem," European Journal of Operational Research, Elsevier, vol. 307(2), pages 842-859.
    8. Philippe Delquié, 2008. "The Value of Information and Intensity of Preference," Decision Analysis, INFORMS, vol. 5(3), pages 129-139, September.
    9. Tianjun Feng & L. Robin Keller, 2006. "A Multiple-Objective Decision Analysis for Terrorism Protection: Potassium Iodide Distribution in Nuclear Incidents," Decision Analysis, INFORMS, vol. 3(2), pages 76-93, June.
    10. Sriwastava, Ambuj & Reichert, Peter, 2023. "Reducing sample size requirements by extending discrete choice experiments to indifference elicitation," Journal of choice modelling, Elsevier, vol. 48(C).
    11. Jin Zhao, 2019. "Information Entropy-Based Housing Spatiotemporal Dependence," The Journal of Real Estate Finance and Economics, Springer, vol. 58(1), pages 21-50, January.
    12. Loomes, Graham, 1995. "The myth of the HYE," Journal of Health Economics, Elsevier, vol. 14(1), pages 1-7, May.
    13. Minardi, Stefania & Savochkin, Andrei, 2015. "Preferences with grades of indecisiveness," Journal of Economic Theory, Elsevier, vol. 155(C), pages 300-331.
    14. Lahdelma, Risto & Makkonen, Simo & Salminen, Pekka, 2009. "Two ways to handle dependent uncertainties in multi-criteria decision problems," Omega, Elsevier, vol. 37(1), pages 79-92, February.
    15. Ralph L. Keeney, 2002. "Common Mistakes in Making Value Trade-Offs," Operations Research, INFORMS, vol. 50(6), pages 935-945, December.
    16. L. Robin Keller & Jay Simon, 2019. "Preference Functions for Spatial Risk Analysis," Risk Analysis, John Wiley & Sons, vol. 39(1), pages 244-256, January.
    17. Andrea C. Hupman & Jay Simon, 2023. "The Legacy of Peter Fishburn: Foundational Work and Lasting Impact," Decision Analysis, INFORMS, vol. 20(1), pages 1-15, March.
    18. William K. Klimack & Jack M. Kloeber & Kenneth W. Bauer & Mark E. Oxley, 2015. "An Empirical Examination of Multiple Objective Risk Attitudes," Decision Analysis, INFORMS, vol. 12(2), pages 96-103, June.
    19. Bous, Géraldine & Fortemps, Philippe & Glineur, François & Pirlot, Marc, 2010. "ACUTA: A novel method for eliciting additive value functions on the basis of holistic preference statements," European Journal of Operational Research, Elsevier, vol. 206(2), pages 435-444, October.
    20. Robert A. Dees & Scott T. Nestler & Robert Kewley, 2013. "WholeSoldier Performance Appraisal to Support Mentoring and Personnel Decisions," Decision Analysis, INFORMS, vol. 10(1), pages 82-97, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:navres:v:58:y:2011:i:3:p:236-254. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1002/(ISSN)1520-6750 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.