IDEAS home Printed from https://ideas.repec.org/a/inm/ordeca/v1y2004i1p35-50.html
   My bibliography  Save this article

A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems

Author

Listed:
  • Jacob W. Ulvila

    (Decision Science Associates, Inc., P.O. Box 969, Vienna, Virginia 22183)

  • John E. Gaffney

    (Lockheed Martin, 700 North Frederick Avenue, Gaithersburg, Maryland 20879)

Abstract

This paper presents a decision analysis method for evaluating computer intrusion detection systems. The method integrates and extends receiver operating characteristic (ROC) and cost analysis methods to provide an expected cost metric. We demonstrate that both the ROC analysis and cost analysis methods are incomplete. Furthermore, we demonstrate how a decision tree can combine and extend the ROC and cost analysis methods to provide an expected cost metric that reflects the intrusion detection system's ROC curve, costs, and assessments of the hostility of the environment as summarized by the prior probability of intrusion. We further demonstrate how this method can be used to decide the optimal operating point on an intrusion detector's ROC curve, choose the best intrusion detection system, compare the value of one intrusion detection system with another's, determine the value of an intrusion detector over no detector, and determine how to adjust the operation of an intrusion detector to respond to changes in its environment. General results are given and the method is illustrated in several numerical examples that involve both hypothetical and real intrusion detection systems. We demonstrate that, contrary to common advice, the value of an intrusion detection system depends not only on its ROC curve, but also on various costs (such as those associated with making incorrect decisions about detection) and the hostility of the operating environment. Conclusions are drawn about the design and evaluation of intrusion detection systems and the role for decision analysis in that design and evaluation.

Suggested Citation

  • Jacob W. Ulvila & John E. Gaffney, 2004. "A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems," Decision Analysis, INFORMS, vol. 1(1), pages 35-50, March.
    • Handle: RePEc:inm:ordeca:v:1:y:2004:i:1:p:35-50
    DOI: 10.1287/deca.1030.0001
    as

    Download full text from publisher

    File URL: http://dx.doi.org/10.1287/deca.1030.0001
    Download Restriction: no
    File URL: https://libkey.io/10.1287/deca.1030.0001?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Huseyin Cavusoglu & Byungwan Koh & Srinivasan Raghunathan, 2010. "An Analysis of the Impact of Passenger Profiling for Transportation Security," Operations Research, INFORMS, vol. 58(5), pages 1287-1302, October.
    2. Xing Gao & Weijun Zhong & Shue Mei, 2013. "Information Security Investment When Hackers Disseminate Knowledge," Decision Analysis, INFORMS, vol. 10(4), pages 352-368, December.
    3. ÇakanyIldIrIm, Metin & Yue, Wei T. & Ryu, Young U., 2009. "The management of intrusion detection: Configuration, inspection, and investment," European Journal of Operational Research, Elsevier, vol. 195(1), pages 186-204, May.
    4. Rakesh K. Sarin & L. Robin Keller, 2013. "From the Editors: Probability Approximations, Anti-Terrorism Strategy, and Bull's-Eye Display for Performance Feedback," Decision Analysis, INFORMS, vol. 10(1), pages 1-5, March.
    5. Huseyin Cavusoglu & Srinivasan Raghunathan & Hasan Cavusoglu, 2009. "Configuration of and Interaction Between Information Security Technologies: The Case of Firewalls and Intrusion Detection Systems," Information Systems Research, INFORMS, vol. 20(2), pages 198-217, June.
    6. Matthias Bogaert & Michel Ballings & Martijn Hosten & Dirk Van den Poel, 2017. "Identifying Soccer Players on Facebook Through Predictive Analytics," Decision Analysis, INFORMS, vol. 14(4), pages 274-297, December.
    7. Huseyin Cavusoglu & Hasan Cavusoglu, 2007. "Assessing the Value of Network Security Technologies: The Impact of Configuration and Interaction on Value," Working Papers 07-19, NET Institute, revised Aug 2007.
    8. Huseyin Cavusoglu & Young Kwark & Bin Mai & Srinivasan Raghunathan, 2013. "Passenger Profiling and Screening for Aviation Security in the Presence of Strategic Attackers," Decision Analysis, INFORMS, vol. 10(1), pages 63-81, March.
    9. Robert T. Clemen & Don N. Kleinmuntz, 2004. "From the Editors…," Decision Analysis, INFORMS, vol. 1(3), pages 129-130, September.
    10. Mehmet Eren Ahsen & Mehmet Ulvi Saygi Ayvaci & Srinivasan Raghunathan, 2019. "When Algorithmic Predictions Use Human-Generated Data: A Bias-Aware Classification Algorithm for Breast Cancer Diagnosis," Service Science, INFORMS, vol. 30(1), pages 97-116, March.
    11. Vijay Mookerjee & Radha Mookerjee & Alain Bensoussan & Wei T. Yue, 2011. "When Hackers Talk: Managing Information Security Under Variable Attack Rates and Knowledge Dissemination," Information Systems Research, INFORMS, vol. 22(3), pages 606-623, September.
    12. Young U. Ryu & Hyeun-Suk Rhee, 2008. "Improving Intrusion Prevention Models: Dual-Threshold and Dual-Filter Approaches," INFORMS Journal on Computing, INFORMS, vol. 20(3), pages 356-367, August.
    13. Bernardino, Wilton & Ospina, Raydonal & Souza, Filipe Costa de & Rêgo, Leandro & Pereira, Felipe, 2021. "Risk curves: A methodology to evaluate the risk of fraud by stock price manipulation based on game theory and detection software," Journal of Economics and Business, Elsevier, vol. 113(C).
    14. Hulisi Ogut & Huseyin Cavusoglu & Srinivasan Raghunathan, 2008. "Intrusion-Detection Policies for IT Security Breaches," INFORMS Journal on Computing, INFORMS, vol. 20(1), pages 112-123, February.
    15. Alain Bensoussan & Vijay Mookerjee & Wei T. Yue, 2020. "Managing Information System Security Under Continuous and Abrupt Deterioration," Production and Operations Management, Production and Operations Management Society, vol. 29(8), pages 1894-1917, August.
    16. Alain Bensoussan & Radha Mookerjee & Vijay Mookerjee & Wei T. Yue, 2009. "Maintaining Diagnostic Knowledge-Based Systems: A Control-Theoretic Approach," Management Science, INFORMS, vol. 55(2), pages 294-310, February.
    17. Yonghua Ji & Subodha Kumar & Vijay Mookerjee, 2016. "When Being Hot Is Not Cool: Monitoring Hot Lists for Information Security," Information Systems Research, INFORMS, vol. 27(4), pages 897-918, December.
    18. Xing Gao & Weijun Zhong & Shue Mei, 2015. "Security investment and information sharing under an alternative security breach probability function," Information Systems Frontiers, Springer, vol. 17(2), pages 423-438, April.
    19. Huseyin Cavusoglu & Srinivasan Raghunathan, 2004. "Configuration of Detection Software: A Comparison of Decision and Game Theory Approaches," Decision Analysis, INFORMS, vol. 1(3), pages 131-148, September.
    20. Fessi, B.A. & Hamdi, M. & Benabdallah, S. & Boudriga, N., 2007. "A decisional framework system for computer network intrusion detection," European Journal of Operational Research, Elsevier, vol. 177(3), pages 1824-1838, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:inm:ordeca:v:1:y:2004:i:1:p:35-50. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Asher (email available below). General contact details of provider: https://edirc.repec.org/data/inforea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.