IDEAS home Printed from https://ideas.repec.org/a/spr/infosf/v19y2017i1d10.1007_s10796-015-9586-x.html
   My bibliography  Save this article

What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation

Author

Listed:
  • Dmytro Babik

    (The University of North Carolina at Greensboro)

  • Rahul Singh

    (The University of North Carolina at Greensboro)

  • Xia Zhao

    (The University of North Carolina at Greensboro)

  • Eric W. Ford

    (Johns Hopkins University
    Social Learning Solutions LLC)

Abstract

Miscalibration, the failure to accurately evaluate one’s own work relative to others' evaluation, is a common concern in social systems of knowledge creation where participants act as both creators and evaluators. Theories of social norming hold that individual’s self-evaluation miscalibration diminishes over multiple iterations of creator-evaluator interactions and shared understanding emerges. This paper explores intersubjectivity and the longitudinal dynamics of miscalibration between creators' and evaluators' assessments in IT-enabled social knowledge creation and refinement systems. Using Latent Growth Modeling, we investigated dynamics of creator’s assessments of their own knowledge artifacts compared to peer evaluators' to determine whether miscalibration attenuates over multiple interactions. Contrary to theory, we found that creator’s self-assessment miscalibration does not attenuate over repeated interactions. Moreover, depending on the degree of difference, we found self-assessment miscalibration to amplify over time with knowledge artifact creators' diverging farther from their peers' collective opinion. Deeper analysis found no significant evidence of the influence of bias and controversy on miscalibration. Therefore, relying on social norming to correct miscalibration in knowledge creation environments (e.g., social media interactions) may not function as expected.

Suggested Citation

  • Dmytro Babik & Rahul Singh & Xia Zhao & Eric W. Ford, 2017. "What you think and what I think: Studying intersubjectivity in knowledge artifacts evaluation," Information Systems Frontiers, Springer, vol. 19(1), pages 31-56, February.
    • Handle: RePEc:spr:infosf:v:19:y:2017:i:1:d:10.1007_s10796-015-9586-x
    DOI: 10.1007/s10796-015-9586-x
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10796-015-9586-x
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10796-015-9586-x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. McKenzie, Craig R. M., 1997. "Underweighting Alternatives and Overconfidence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 71(2), pages 141-160, August.
    2. Bobby L. Jones & Daniel S. Nagin & Kathryn Roeder, 2001. "A SAS Procedure Based on Mixture Models for Estimating Developmental Trajectories," Sociological Methods & Research, , vol. 29(3), pages 374-393, February.
    3. Ryvkin, Dmitry & Krajč, Marian & Ortmann, Andreas, 2012. "Are the unskilled doomed to remain unaware?," Journal of Economic Psychology, Elsevier, vol. 33(5), pages 1012-1031.
    4. Stefano Mizzaro, 2003. "Quality control in scholarly publishing: A new proposal," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 54(11), pages 989-1005, September.
    5. Wei-Feng Tung & Soe-Tsyr Yuan & Yen-Cheng Wu & Patrick Hung, 2014. "Collaborative service system design for music content creation," Information Systems Frontiers, Springer, vol. 16(2), pages 291-302, April.
    6. John Seely Brown & Paul Duguid, 2001. "Knowledge and Organization: A Social-Practice Perspective," Organization Science, INFORMS, vol. 12(2), pages 198-213, April.
    7. Sharp, Glen L. & Cutler, Brian L. & Penrod, Steven D., 1988. "Performance feedback improves the resolution of confidence judgments," Organizational Behavior and Human Decision Processes, Elsevier, vol. 42(3), pages 271-283, December.
    8. Zhiqiang (Eric) Zheng & Paul A. Pavlou & Bin Gu, 2014. "Latent Growth Modeling for Information Systems: Theoretical Extensions and Practical Applications," Information Systems Research, INFORMS, vol. 25(3), pages 547-568, September.
    9. Meyer, Bruce D, 1995. "Natural and Quasi-experiments in Economics," Journal of Business & Economic Statistics, American Statistical Association, vol. 13(2), pages 151-161, April.
    10. Shaila M. Miranda & Carol S. Saunders, 2003. "The Social Construction of Meaning: An Alternative Perspective on Information Sharing," Information Systems Research, INFORMS, vol. 14(1), pages 87-106, March.
    11. Minhong Wang, 2011. "Integrating organizational, social, and individual perspectives in Web 2.0-based workplace e-learning," Information Systems Frontiers, Springer, vol. 13(2), pages 191-205, April.
    12. Sieck, Winston R. & Merkle, Edgar C. & Van Zandt, Trisha, 2007. "Option fixation: A cognitive contributor to overconfidence," Organizational Behavior and Human Decision Processes, Elsevier, vol. 103(1), pages 68-83, May.
    13. Amin, Ash & Roberts, Joanne, 2008. "Knowing in action: Beyond communities of practice," Research Policy, Elsevier, vol. 37(2), pages 353-369, March.
    14. Stone, Eric R. & Opel, Ryan B., 2000. "Training to Improve Calibration and Discrimination: The Effects of Performance and Environmental Feedback," Organizational Behavior and Human Decision Processes, Elsevier, vol. 83(2), pages 282-309, November.
    15. Kreps, David M, 1997. "Intrinsic Motivation and Extrinsic Incentives," American Economic Review, American Economic Association, vol. 87(2), pages 359-364, May.
    16. M. Lynne Markus & Daniel Robey, 1988. "Information Technology and Organizational Change: Causal Structure in Theory and Research," Management Science, INFORMS, vol. 34(5), pages 583-598, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ryvkin, Dmitry & Krajč, Marian & Ortmann, Andreas, 2012. "Are the unskilled doomed to remain unaware?," Journal of Economic Psychology, Elsevier, vol. 33(5), pages 1012-1031.
    2. Fellner-Röhling, Gerlinde & Hromek, Kristijan & Kleinknecht, Janina & Ludwig, Sandra, 2023. "How to counteract biased self-assessments? An experimental investigation of reactions to social information," Journal of Economic Behavior & Organization, Elsevier, vol. 206(C), pages 1-25.
    3. Kuebart, Andreas & Ibert, Oliver, 2019. "Beyond territorial conceptions of entrepreneurial ecosystems: The dynamic spatiality of knowledge brokering in seed accelerators," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 63(2-4), pages 118-133.
    4. Bolger, Fergus & Onkal-Atay, Dilek, 2004. "The effects of feedback on judgmental interval predictions," International Journal of Forecasting, Elsevier, vol. 20(1), pages 29-39.
    5. Sandra Dubouloz & Anne Berthinier-Poncet & Luciana Castro Gonçalves & Emilie Ruiz & Catherine Thevenard-Puthod, 2021. "Innovation communities: from their characterization to the questioning of their boundaries [Comunidades de innovación: desde su caracterización hasta el cuestionamiento de sus fronteras]," Post-Print hal-02891869, HAL.
    6. Müller, Felix Claus & Ibert, Oliver, 2014. "(Re-)Sources of Innovation: Understanding and Comparing Innovation Dynamics through the Lens of Communities of Practice," IRS Working Papers 52, Leibniz Institute for Research on Society and Space (IRS).
    7. Ibert, Oliver & Müller, Felix C., 2015. "Network dynamics in constellations of cultural differences: Relational distance in innovation processes in legal services and biotechnology," Research Policy, Elsevier, vol. 44(1), pages 181-194.
    8. Frößler, F. & Rukanova, B. & Klein, F., 2008. "Inter-Organisational Network Formation and Sense-Making : Initiation and Management of a Living Lab," Serie Research Memoranda 0017, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    9. Marijn A. van Weele & Henk J. Steinz & Frank J. van Rijnsoever, 2018. "Start‐up Communities as Communities of Practice: Shining a Light on Geographical Scale and Membership," Tijdschrift voor Economische en Sociale Geografie, Royal Dutch Geographical Society KNAG, vol. 109(2), pages 173-188, April.
    10. Ferguson, J.E. & Huysman, M.H., 2009. "Between ambition and approach: towards sustainable knowledge management in development organizations," Serie Research Memoranda 0003, VU University Amsterdam, Faculty of Economics, Business Administration and Econometrics.
    11. Ramkumar Janakiraman & Eunho Park & Emre M. Demirezen & Subodha Kumar, 2023. "The Effects of Health Information Exchange Access on Healthcare Quality and Efficiency: An Empirical Investigation," Management Science, INFORMS, vol. 69(2), pages 791-811, February.
    12. Ravi Patnayakuni & Cynthia P. Ruppel, 2010. "A socio-technical approach to improving the systems development process," Information Systems Frontiers, Springer, vol. 12(2), pages 219-234, April.
    13. Ivory, Chris & Casey, Rebecca & Watson, Kayleigh, 2016. "The role of mobile ICT in repair worker communities of practice," 27th European Regional ITS Conference, Cambridge (UK) 2016 148676, International Telecommunications Society (ITS).
    14. Thomas O. Meservy & Matthew L. Jensen & Kelly J. Fadel, 2014. "Evaluation of Competing Candidate Solutions in Electronic Networks of Practice," Information Systems Research, INFORMS, vol. 25(1), pages 15-34, March.
    15. Matias Ramirez & Xibao Li, 2008. "Learning and sharing in a Chinese high-technology cluster: A study of inter-firm and intra-firm knowledge flows between R&D employees," SPRU Working Paper Series 174, SPRU - Science Policy Research Unit, University of Sussex Business School.
    16. Kilfoyle, Eksa & Richardson, Alan J. & MacDonald, Laura D., 2013. "Vernacular accountings: Bridging the cognitive and the social in the analysis of employee-generated accounting systems," Accounting, Organizations and Society, Elsevier, vol. 38(5), pages 382-396.
    17. Marlous Agterberg & Bart Van Den Hooff & Marleen Huysman & Maura Soekijad, 2010. "Keeping the Wheels Turning: The Dynamics of Managing Networks of Practice," Journal of Management Studies, Wiley Blackwell, vol. 47(1), pages 85-108, January.
    18. Ferguson, Julie & Huysman, Marleen & Soekijad, Maura, 2010. "Knowledge Management in Practice: Pitfalls and Potentials for Development," World Development, Elsevier, vol. 38(12), pages 1797-1810, December.
    19. Sandra Dubouloz & Anne Berthinier-Poncet & Luciana Castro Gonçalves & Emilie Ruiz & Catherine Thevenard-Puthod, 2020. "Communautés d'innovation : de leur caractérisation au questionnement de leurs frontières," Working Papers hal-02891869, HAL.
    20. Windschitl, Paul D. & Scherer, Aaron M. & Smith, Andrew R. & Rose, Jason P., 2013. "Why so confident? The influence of outcome desirability on selective exposure and likelihood judgment," Organizational Behavior and Human Decision Processes, Elsevier, vol. 120(1), pages 73-86.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:infosf:v:19:y:2017:i:1:d:10.1007_s10796-015-9586-x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.