IDEAS home Printed from https://ideas.repec.org/a/eee/reensy/v197y2020ics0951832019301097.html
   My bibliography  Save this article

On reliability assessment when a software-based system is replaced by a thought-to-be-better one

Author

Listed:
  • Littlewood, Bev
  • Salako, Kizito
  • Strigini, Lorenzo
  • Zhao, Xingyu

Abstract

The failure history of pre-existing systems can inform a reliability assessment of a new system. Such assessments – consisting of arguments based on evidence from older systems – are attractive and have been used for quite some time for, typically, mechanical/hardware-only systems. But their application to software-based systems brings some challenges. In this paper, we present a conservative, Bayesian approach to software reliability assessment – one that combines reliability evidence from an old system with an assessor’s confidence in a newer system being an improved replacement for the old one. We demonstrate, via different scenarios, what a thought-to-be-better replacement formally means in practice, and what it allows one to believe about actual reliability improvement. The results can be used directly in a reliability assessment, or to caution system stakeholders and industry regulators against using other models that give optimistic assessments. For instance, even if one is certain that some new software must be more reliable than an old product, using the reliability distribution for the old software as a prior distribution when assessing the new system gives optimistic, not conservative, predictions for the posterior reliability of the new system after seeing operational testing evidence.

Suggested Citation

  • Littlewood, Bev & Salako, Kizito & Strigini, Lorenzo & Zhao, Xingyu, 2020. "On reliability assessment when a software-based system is replaced by a thought-to-be-better one," Reliability Engineering and System Safety, Elsevier, vol. 197(C).
    • Handle: RePEc:eee:reensy:v:197:y:2020:i:c:s0951832019301097
    DOI: 10.1016/j.ress.2019.106752
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0951832019301097
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ress.2019.106752?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. B. Littlewood & J. L. Verrall, 1973. "A Bayesian Reliability Growth Model for Computer Software," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 22(3), pages 332-346, November.
    2. Zhao, Xingyu & Littlewood, Bev & Povyakalo, Andrey & Strigini, Lorenzo & Wright, David, 2018. "Conservative claims for the probability of perfection of a software-based system using operational experience of previous similar systems," Reliability Engineering and System Safety, Elsevier, vol. 175(C), pages 265-282.
    3. Zhao, Xingyu & Littlewood, Bev & Povyakalo, Andrey & Strigini, Lorenzo & Wright, David, 2017. "Modeling the probability of failure on demand (pfd) of a 1-out-of-2 system in which one channel is “quasi-perfectâ€," Reliability Engineering and System Safety, Elsevier, vol. 158(C), pages 230-245.
    4. Bunea, C. & Charitos, T. & Cooke, R.M. & Becker, G., 2005. "Two-stage Bayesian models—application to ZEDB project," Reliability Engineering and System Safety, Elsevier, vol. 90(2), pages 123-130.
    5. Charalambos D. Aliprantis & Kim C. Border, 2006. "Infinite Dimensional Analysis," Springer Books, Springer, edition 0, number 978-3-540-29587-7, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ajit Kumar Behera & Mrutyunjaya Panda & Satchidananda Dehuri, 2021. "Software reliability prediction by recurrent artificial chemical link network," International Journal of System Assurance Engineering and Management, Springer;The Society for Reliability, Engineering Quality and Operations Management (SREQOM),India, and Division of Operation and Maintenance, Lulea University of Technology, Sweden, vol. 12(6), pages 1308-1321, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zhao, Xingyu & Littlewood, Bev & Povyakalo, Andrey & Strigini, Lorenzo & Wright, David, 2018. "Conservative claims for the probability of perfection of a software-based system using operational experience of previous similar systems," Reliability Engineering and System Safety, Elsevier, vol. 175(C), pages 265-282.
    2. Popov, Peter, 2021. "Conservative reliability assessment of a 2-channel software system when one of the channels is probably perfect," Reliability Engineering and System Safety, Elsevier, vol. 216(C).
    3. Campi, Luciano & Zabaljauregui, Diego, 2020. "Optimal market making under partial information with general intensities," LSE Research Online Documents on Economics 104612, London School of Economics and Political Science, LSE Library.
    4. Kaido, Hiroaki, 2017. "Asymptotically Efficient Estimation Of Weighted Average Derivatives With An Interval Censored Variable," Econometric Theory, Cambridge University Press, vol. 33(5), pages 1218-1241, October.
    5. Andrea Attar & Thomas Mariotti & François Salanié, 2021. "Entry-Proofness and Discriminatory Pricing under Adverse Selection," American Economic Review, American Economic Association, vol. 111(8), pages 2623-2659, August.
    6. Qing Tian & Chun-Wu Yeh & Chih-Chiang Fang, 2022. "Bayesian Decision Making of an Imperfect Debugging Software Reliability Growth Model with Consideration of Debuggers’ Learning and Negligence Factors," Mathematics, MDPI, vol. 10(10), pages 1-21, May.
    7. Askoura, Youcef & Billot, Antoine, 2021. "Social decision for a measure society," Journal of Mathematical Economics, Elsevier, vol. 94(C).
    8. Xiaohong Chen & Andres Santos, 2018. "Overidentification in Regular Models," Econometrica, Econometric Society, vol. 86(5), pages 1771-1817, September.
    9. He, Wei & Sun, Yeneng, 2013. "Stationary Markov Perfect Equilibria in Discounted Stochastic Games," MPRA Paper 51274, University Library of Munich, Germany.
    10. Duggan, John, 2011. "General conditions for the existence of maximal elements via the uncovered set," Journal of Mathematical Economics, Elsevier, vol. 47(6), pages 755-759.
    11. Eduardo Perez & Delphine Prady, 2012. "Complicating to Persuade?," Working Papers hal-03583827, HAL.
    12. Romain Blanchard & Laurence Carassus & Miklós Rásonyi, 2018. "No-arbitrage and optimal investment with possibly non-concave utilities: a measure theoretical approach," Mathematical Methods of Operations Research, Springer;Gesellschaft für Operations Research (GOR);Nederlands Genootschap voor Besliskunde (NGB), vol. 88(2), pages 241-281, October.
    13. René Aïd & Matteo Basei & Giorgia Callegaro & Luciano Campi & Tiziano Vargiolu, 2020. "Nonzero-Sum Stochastic Differential Games with Impulse Controls: A Verification Theorem with Applications," Mathematics of Operations Research, INFORMS, vol. 45(1), pages 205-232, February.
    14. He, Wei & Yannelis, Nicholas C., 2015. "Equilibrium theory under ambiguity," Journal of Mathematical Economics, Elsevier, vol. 61(C), pages 86-95.
    15. Light, Bar & Weintraub, Gabriel, 2018. "Mean Field Equilibrium: Uniqueness, Existence, and Comparative Statics," Research Papers 3731, Stanford University, Graduate School of Business.
    16. Massimiliano Amarante & Mario Ghossoub & Edmund Phelps, 2012. "Contracting for Innovation under Knightian Uncertainty," Cahiers de recherche 18-2012, Centre interuniversitaire de recherche en économie quantitative, CIREQ.
    17. Jeongwoo Lee & Jaeok Park, 2019. "Preemptive Entry in Sequential Auctions with Participation Cost," Working papers 2019rwp-141, Yonsei University, Yonsei Economics Research Institute.
    18. Sudhir A. Shah, 2016. "The Generalized Arrow-Pratt Coefficient," Working Papers id:10795, eSocialSciences.
    19. Luçon, Eric, 2020. "Quenched asymptotics for interacting diffusions on inhomogeneous random graphs," Stochastic Processes and their Applications, Elsevier, vol. 130(11), pages 6783-6842.
    20. Lashi Bandara & Paul Bryan, 2020. "Heat kernels and regularity for rough metrics on smooth manifolds," Mathematische Nachrichten, Wiley Blackwell, vol. 293(12), pages 2255-2270, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:reensy:v:197:y:2020:i:c:s0951832019301097. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/reliability-engineering-and-system-safety .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.