IDEAS home Printed from https://ideas.repec.org/a/spr/elmark/v32y2022i4d10.1007_s12525-022-00606-3.html
   My bibliography  Save this article

A nascent design theory for explainable intelligent systems

Author

Listed:
  • Lukas-Valentin Herm

    (Julius-Maximilians-Universität Würzburg)

  • Theresa Steinbach

    (Julius-Maximilians-Universität Würzburg)

  • Jonas Wanner

    (Julius-Maximilians-Universität Würzburg)

  • Christian Janiesch

    (TU Dortmund University)

Abstract

Due to computational advances in the past decades, so-called intelligent systems can learn from increasingly complex data, analyze situations, and support users in their decision-making to address them. However, in practice, the complexity of these intelligent systems renders the user hardly able to comprehend the inherent decision logic of the underlying machine learning model. As a result, the adoption of this technology, especially for high-stake scenarios, is hampered. In this context, explainable artificial intelligence offers numerous starting points for making the inherent logic explainable to people. While research manifests the necessity for incorporating explainable artificial intelligence into intelligent systems, there is still a lack of knowledge about how to socio-technically design these systems to address acceptance barriers among different user groups. In response, we have derived and evaluated a nascent design theory for explainable intelligent systems based on a structured literature review, two qualitative expert studies, a real-world use case application, and quantitative research. Our design theory includes design requirements, design principles, and design features covering the topics of global explainability, local explainability, personalized interface design, as well as psychological/emotional factors.

Suggested Citation

  • Lukas-Valentin Herm & Theresa Steinbach & Jonas Wanner & Christian Janiesch, 2022. "A nascent design theory for explainable intelligent systems," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2185-2205, December.
    • Handle: RePEc:spr:elmark:v:32:y:2022:i:4:d:10.1007_s12525-022-00606-3
    DOI: 10.1007/s12525-022-00606-3
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s12525-022-00606-3
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s12525-022-00606-3?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Patrick Zschech & Richard Horn & Daniel Höschele & Christian Janiesch & Kai Heinrich, 2020. "Intelligent User Assistance for Automated Data Mining Method Selection," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 62(3), pages 227-247, June.
    2. Scott Thiebes & Sebastian Lins & Ali Sunyaev, 2021. "Trustworthy artificial intelligence," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(2), pages 447-464, June.
    3. Christian Janiesch & Patrick Zschech & Kai Heinrich, 2021. "Machine learning and deep learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 685-695, September.
    4. Benedikt Berger & Martin Adam & Alexander Rühr & Alexander Benlian, 2021. "Watch Me Improve—Algorithm Aversion and Demonstrating the Ability to Learn," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 63(1), pages 55-68, February.
    5. Jonas Wanner & Lukas-Valentin Herm & Kai Heinrich & Christian Janiesch, 2022. "The effect of transparency and trust on intelligent system acceptance: Evidence from a user-based study," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2079-2102, December.
    6. Allen S. Lee & Richard L. Baskerville, 2003. "Generalizing Generalizability in Information Systems Research," Information Systems Research, INFORMS, vol. 14(3), pages 221-243, September.
    7. Ahmad Kamal Mohd Nor & Srinivasa Rao Pedapati & Masdi Muhammad & Víctor Leiva, 2022. "Abnormality Detection and Failure Prediction Using Explainable Bayesian Deep Learning: Methodology and Case Study with Industrial Data," Mathematics, MDPI, vol. 10(4), pages 1-37, February.
    8. Babak Abedin & Christian Meske & Iris Junglas & Fethi Rabhi & Hamid R. Motahari-Nezhad, 2022. "Designing and Managing Human-AI Interactions," Information Systems Frontiers, Springer, vol. 24(3), pages 691-697, June.
    9. Mahmud, Hasan & Islam, A.K.M. Najmul & Ahmed, Syed Ishtiaque & Smolander, Kari, 2022. "What influences algorithmic decision-making? A systematic literature review on algorithm aversion," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    10. Shin, Donghee & Zhong, Bu & Biocca, Frank A., 2020. "Beyond user experience: What constitutes algorithmic experiences?," International Journal of Information Management, Elsevier, vol. 52(C).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. AKM Bahalul Haque & Najmul Islam & Patrick Mikalef, 2025. "To Explain or Not To Explain: An Empirical Investigation of AI-based Recommendations on Social Media Platforms," Electronic Markets, Springer;IIM University of St. Gallen, vol. 35(1), pages 1-18, December.
    2. Christian Meske & Babak Abedin & Mathias Klier & Fethi Rabhi, 2022. "Explainable and responsible artificial intelligence," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2103-2106, December.
    3. Alexander Mayr & Philip Stahmann & Maximilian Nebel & Christian Janiesch, 2024. "Still doing it yourself? Investigating determinants for the adoption of intelligent process automation," Electronic Markets, Springer;IIM University of St. Gallen, vol. 34(1), pages 1-22, December.
    4. Rainer Alt, 2022. "Electronic Markets on AI and standardization," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 1795-1805, December.
    5. Pascal Fechner & Fabian König & Jannik Lockl & Maximilian Röglinger, 2024. "How Artificial Intelligence Challenges Tailorable Technology Design," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 66(3), pages 357-376, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jonas Wanner & Lukas-Valentin Herm & Kai Heinrich & Christian Janiesch, 2022. "The effect of transparency and trust on intelligent system acceptance: Evidence from a user-based study," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2079-2102, December.
    2. Alexander Mayr & Philip Stahmann & Maximilian Nebel & Christian Janiesch, 2024. "Still doing it yourself? Investigating determinants for the adoption of intelligent process automation," Electronic Markets, Springer;IIM University of St. Gallen, vol. 34(1), pages 1-22, December.
    3. Mahmud, Hasan & Islam, A.K.M. Najmul & Mitra, Ranjan Kumar, 2023. "What drives managers towards algorithm aversion and how to overcome it? Mitigating the impact of innovation resistance through technology readiness," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    4. Patrick Zschech, 2023. "Beyond descriptive taxonomies in data analytics: a systematic evaluation approach for data-driven method pipelines," Information Systems and e-Business Management, Springer, vol. 21(1), pages 193-227, March.
    5. Matti Minkkinen & Markus Philipp Zimmer & Matti Mäntymäki, 2023. "Co-Shaping an Ecosystem for Responsible AI: Five Types of Expectation Work in Response to a Technological Frame," Information Systems Frontiers, Springer, vol. 25(1), pages 103-121, February.
    6. Niklas Kühl & Max Schemmer & Marc Goutier & Gerhard Satzger, 2022. "Artificial intelligence and machine learning," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2235-2244, December.
    7. Roman Lukyanenko & Wolfgang Maass & Veda C. Storey, 2022. "Trust in artificial intelligence: From a Foundational Trust Framework to emerging research opportunities," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 1993-2020, December.
    8. Hermann, Erik & Puntoni, Stefano, 2024. "Artificial intelligence and consumer behavior: From predictive to generative AI," Journal of Business Research, Elsevier, vol. 180(C).
    9. Md Shajalal & Alexander Boden & Gunnar Stevens, 2022. "Explainable product backorder prediction exploiting CNN: Introducing explainable models in businesses," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 2107-2122, December.
    10. Rainer Alt, 2021. "Electronic Markets on digital platforms and AI," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(2), pages 233-241, June.
    11. Jen-Yu Lee & Tien-Thinh Nguyen & Hong-Giang Nguyen & Jen-Yao Lee, 2022. "Towards Predictive Crude Oil Purchase: A Case Study in the USA and Europe," Energies, MDPI, vol. 15(11), pages 1-15, May.
    12. Gunasekaran, Angappa & Irani, Zahir & Choy, King-Lun & Filippi, Lionel & Papadopoulos, Thanos, 2015. "Performance measures and metrics in outsourcing decisions: A review for research and applications," International Journal of Production Economics, Elsevier, vol. 161(C), pages 153-166.
    13. Yongping Bao & Ludwig Danwitz & Fabian Dvorak & Sebastian Fehrler & Lars Hornuf & Hsuan Yu Lin & Bettina von Helversen, 2022. "Similarity and Consistency in Algorithm-Guided Exploration," CESifo Working Paper Series 10188, CESifo.
    14. Mostafa Bigdeli & Mahsa Akbari, 2024. "Machine-learning-based Classification of Customers’ Behavioural Model in Instagram," Paradigm, , vol. 28(2), pages 223-240, December.
    15. Emmanuelle Vaast & Geoff Walsham, 2009. "Trans-Situated Learning: Supporting a Network of Practice with an Information Infrastructure," Information Systems Research, INFORMS, vol. 20(4), pages 547-564, December.
    16. Eduard Hartwich & Alexander Rieger & Johannes Sedlmeir & Dominik Jurek & Gilbert Fridgen, 2023. "Machine economies," Electronic Markets, Springer;IIM University of St. Gallen, vol. 33(1), pages 1-13, December.
    17. Rainer Alt, 2021. "Electronic Markets on robotics," Electronic Markets, Springer;IIM University of St. Gallen, vol. 31(3), pages 465-471, September.
    18. Arsenyan, Jbid & Mirowska, Agata & Piepenbrink, Anke, 2023. "Close encounters with the virtual kind: Defining a human-virtual agent coexistence framework," Technological Forecasting and Social Change, Elsevier, vol. 193(C).
    19. Michael Vössing & Niklas Kühl & Matteo Lind & Gerhard Satzger, 2022. "Designing Transparency for Effective Human-AI Collaboration," Information Systems Frontiers, Springer, vol. 24(3), pages 877-895, June.
    20. Zhewei Zhang & Youngjin Yoo & Kalle Lyytinen & Aron Lindberg, 2021. "The Unknowability of Autonomous Tools and the Liminal Experience of Their Use," Information Systems Research, INFORMS, vol. 32(4), pages 1192-1213, December.

    More about this item

    Keywords

    Artificial intelligence; Explainable artificial intelligence; XAI; Design science research; Design theory; Intelligent systems;
    All these keywords.

    JEL classification:

    • C6 - Mathematical and Quantitative Methods - - Mathematical Methods; Programming Models; Mathematical and Simulation Modeling
    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • C9 - Mathematical and Quantitative Methods - - Design of Experiments
    • M15 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Business Administration - - - IT Management

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:elmark:v:32:y:2022:i:4:d:10.1007_s12525-022-00606-3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.