Exploring Human and Artificial Intelligence Collaboration and Its Impact on Organizational Performance: A Multi-Level Analysis

To achieve great performance and ensure their long-term survival, organizations must successfully act in and adapt to the reality that surrounds them, which requires organizations to learn effectively. For decades, organizations have relied exclusively on human learning for this purpose. With today’s rise of machine learning (ML) systems as a modern form of artificial intelligence (AI) and their ability to autonomously learn and act, ML systems can now also contribute to this vital process, offering organizations an alternative way to learn. Although organizations are increasingly adopting ML systems within a wide range of processes, we still know surprisingly little about how the learning of humans and ML systems affects each other and how their mutual learning affects organizational performance. Although a significant amount of research has addressed ML, existing research leaves it largely unclear whether and when humans and ML systems act as beneficial complementarities or as mutual impediments within the context of learning. This is problematic, as the (mis)use of ML systems may corrupt an organization’s central process of learning and thus impair the organizational adaptation that is crucial for organizational survival. To help organizations facilitate useful synergies of humans and ML systems, this dissertation explores humans’ and ML systems’ idiosyncrasies and their bilateral interplay. As research on organizational learning has demonstrated, the key to managing such dynamics is the effective coordination of the ones who learn. The studies that were conducted for this dissertation therefore aim to uncover virtuous and vicious dynamics between humans and ML systems and how these dynamics can be managed to increase organizational performance. To take a holistic perspective, this dissertation explores three central levels of analysis. The first level of analysis deals with performance impacts on the individual level. Here, the analysis focuses on two essential issues. First, the availability of ML systems as an alternative to humans requires organizations to rethink their problem delegation strategies. Organizations can benefit the most from the relative strengths of humans and ML systems if they are able to delegate problems to those whose expertise and capabilities best fit the problem. This requires organizations to develop an understanding of the problem characteristics that point to problems that are better (or less) suited to being solved by ML systems than by humans. Using a qualitative interview approach, the first study identifies central criteria and procedural artifacts and synthesizes these into a framework for identifying and evaluating problems in ML contexts. The framework provides a theoretical basis to help inform research about delegation decisions between humans and ML systems by unpacking problem nuances that decisively render problems suitable for ML systems. Building on these insights, a subsequent qualitative analysis explores how the dependency between a human and an ML system with respect to the delegated problem affects performance outcomes. The theoretical model that is proposed explains individual performance gains that result from ML systems’ use as a function of the fit between task, data, and technology characteristics. The model highlights how idiosyncrasies of an ML system can affect a human expert’s task execution performance when the expert bases her/his task execution on the ML system’s contributions. This study provides first empirical evidence on controllable levers for managing involved dependencies to increase individual performance. The second level of analysis focuses on performance impacts on the group level. In contrast to traditional (non-ML) information systems, ML systems’ unique learning ability enables them to contribute independently to team endeavors, joining groups as active members that can affect group dynamics through their own contributions. Thus, in a third study, a digital trace analysis is conducted to explore the dynamics of a real-world case in which a group of human traders and a productively trading reinforcement ML system collaborate during trading. The studied case reveals that bilateral learning between multiple humans and an ML system can increase trading performance, which appears to be the result of an emerging virtuous cycle between the humans and the ML system. The findings demonstrate that the interactions between the humans and the ML system can lead to group performance that outperforms the individual trading of either the humans or the ML system. However, in order to achieve this, organizations must effectively coordinate the knowledge transfer and the roles of the involved humans and the ML system. The third level of analysis focuses on performance impacts on the organization level. As ML systems increasingly contribute to organizational processes in all areas of the organization, changes in the organization’s fundamental concepts are likely to occur, and these may affect the organization’s overall performance. In a fourth study, a series of agent-based simulations are therefore used to explore the dynamics of organization-wide interactions between humans and ML systems. The results imply that ML systems can help stimulate the pursuit of innovative directions, liberating humans from exploring unorthodox ideas. The results also show that the alignment of human learning and ML is largely beneficial but can, under certain conditions, become detrimental to organizations. The findings emphasize that effective coordination of humans and ML systems that takes environmental conditions into account can determine the positive and negative impacts of ML systems on organization-level performance. The analyses included in this dissertation highlight that it is precisely the unique differences between humans and ML systems that often seem to make them better complements than substitutes for one another. The secret to unleashing the true potential of ML systems may therefore lie in effectively coordinating the differences between humans and ML systems within their bilateral relationship to produce virtuous cycles of mutual improvement. This dissertation is a first step toward developing theory and guidance on coordinating the dynamics between humans and ML systems, with the aim of helping to rethink collaboration theory in the era of AI.