IDEAS home Printed from https://ideas.repec.org/a/eee/ijoais/v47y2022ics1467089522000306.html
   My bibliography  Save this article

An eye tracking experiment investigating synonymy in conceptual model validation

Author

Listed:
  • Boot, Walter R.
  • Dunn, Cheryl L.
  • Fulmer, Bachman P.
  • Gerard, Gregory J.
  • Grabski, Severin V.

Abstract

A key advantage of conceptual models is that their quality can be evaluated and validated before beginning the costlier stages of information system development. Few research studies investigate the validation process for such models, particularly regarding multiplicities, even though multiplicity mistakes can be very costly. We investigated the validation of conceptual model multiplicities, varying how closely natural language statements of business rules match the models that purport to represent those rules. Participants in an eye tracking experiment completed validation tasks in which they viewed a statement and an accompanying UML class diagram in which a specified multiplicity was consistent with the statement (valid) or inconsistent with the statement (invalid). We varied whether the focal multiplicity was a minimum or a maximum and varied the class diagram’s semantics and order compared to that of the statement. Logistic regression was used to analyze the relationship between accuracy and the experimental manipulations and controls. The results show that the odds of accuracy in validating class diagrams that used synonyms instead of the exact statement terminology were only 0.46 times the odds of accuracy when the class diagram and statement words matched, showing a costly effect of synonymy. Interestingly, independent of the three levels of relative semantics, the odds of accuracy were 0.48 times when class diagrams were consistent with business rules as they were when class diagrams were inconsistent with business rules. To gain insight into cognition under correct task performance, we conducted additional linear regression analysis on various eye tracking metrics for only the accurate responses. Again, synonymy was observed to be costly, with a cognitive burden of increased integrative transitions between statement and model in the range of 39 to 66%.

Suggested Citation

  • Boot, Walter R. & Dunn, Cheryl L. & Fulmer, Bachman P. & Gerard, Gregory J. & Grabski, Severin V., 2022. "An eye tracking experiment investigating synonymy in conceptual model validation," International Journal of Accounting Information Systems, Elsevier, vol. 47(C).
  • Handle: RePEc:eee:ijoais:v:47:y:2022:i:c:s1467089522000306
    DOI: 10.1016/j.accinf.2022.100578
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1467089522000306
    Download Restriction: Full text for ScienceDirect subscribers only

    File URL: https://libkey.io/10.1016/j.accinf.2022.100578?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. François Bodart & Arvind Patel & Marc Sim & Ron Weber, 2001. "Should Optional Properties Be Used in Conceptual Modelling? A Theory and Three Empirical Tests," Information Systems Research, INFORMS, vol. 12(4), pages 384-405, December.
    2. Brian Dobing & Jeffrey Parsons, 2008. "Dimensions of UML Diagram Use: A Survey of Practitioners," Journal of Database Management (JDM), IGI Global, vol. 19(1), pages 1-18, January.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ben Roelens & Geert Poels, 2015. "The Development and Experimental Evaluation of a Focused Business Model Representation," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 57(1), pages 61-71, February.
    2. A. Maes & G. Poels, 2006. "Development of a user evaluations based quality model for conceptual modeling," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 06/406, Ghent University, Faculty of Economics and Business Administration.
    3. Paul L. Bowen & Robert A. O'Farrell & Fiona H. Rohde, 2009. "An Empirical Investigation of End-User Query Development: The Effects of Improved Model Expressiveness vs. Complexity," Information Systems Research, INFORMS, vol. 20(4), pages 565-584, December.
    4. Roger Clarke & Andrew Burton-Jones & Ron Weber, 2016. "On the Ontological Quality and Logical Quality of Conceptual-Modeling Grammars: The Need for a Dual Perspective," Information Systems Research, INFORMS, vol. 27(2), pages 365-382, June.
    5. Yair Wand & Ron Weber, 2002. "Research Commentary: Information Systems and Conceptual Modeling—A Research Agenda," Information Systems Research, INFORMS, vol. 13(4), pages 363-376, December.
    6. Jan Mendling & Jan Recker & Hajo A. Reijers & Henrik Leopold, 2019. "An Empirical Review of the Connection Between Model Viewer Characteristics and the Comprehension of Conceptual Process Models," Information Systems Frontiers, Springer, vol. 21(5), pages 1111-1135, October.
    7. Andrew Burton-Jones & Peter N. Meso, 2006. "Conceptualizing Systems for Understanding: An Empirical Test of Decomposition Principles in Object-Oriented Analysis," Information Systems Research, INFORMS, vol. 17(1), pages 38-60, March.
    8. Vijay Khatri & Iris Vessey & V. Ramesh & Paul Clay & Sung-Jin Park, 2006. "Understanding Conceptual Schemas: Exploring the Role of Application and IS Domain Knowledge," Information Systems Research, INFORMS, vol. 17(1), pages 81-99, March.
    9. G. Poels & A. Maes & F. Gailly & R. Paemeleire, 2004. "User Comprehension of Accounting Information Structures: An Empirical Test of the REA Model," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 04/254, Ghent University, Faculty of Economics and Business Administration.
    10. Roman Lukyanenko & Wolfgang Maass & Veda C. Storey, 2022. "Trust in artificial intelligence: From a Foundational Trust Framework to emerging research opportunities," Electronic Markets, Springer;IIM University of St. Gallen, vol. 32(4), pages 1993-2020, December.
    11. Oktay Turetken & Ahmet Dikici & Irene Vanderfeesten & Tessa Rompen & Onur Demirors, 2020. "The Influence of Using Collapsed Sub-processes and Groups on the Understandability of Business Process Models," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 62(2), pages 121-141, April.
    12. Andreas L. Opdahl & Brian Henderson-Sellers, 2004. "A Template for Defining Enterprise Modelling Constructs," Journal of Database Management (JDM), IGI Global, vol. 15(2), pages 39-73, April.
    13. Guan, Jian & Levitan, Alan S. & Kuhn, John R., 2013. "How AIS can progress along with ontology research in IS," International Journal of Accounting Information Systems, Elsevier, vol. 14(1), pages 21-38.
    14. G. Poels & A. Maes & F. Gailly & R. Paemeleire, 2004. "The Pragmatic Quality of Resources-Events-Agents Diagrams: An Experimental Evaluation," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 04/219, Ghent University, Faculty of Economics and Business Administration.
    15. Sebastian Schlauderer & Sven Overhage, 2018. "BoSDL: An Approach to Describe the Business Logic of Software Services in Domain-Specific Terms," Business & Information Systems Engineering: The International Journal of WIRTSCHAFTSINFORMATIK, Springer;Gesellschaft für Informatik e.V. (GI), vol. 60(5), pages 393-413, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:ijoais:v:47:y:2022:i:c:s1467089522000306. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: https://www.journals.elsevier.com/international-journal-of-accounting-information-systems/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.