Models that learn how humans learn: The case of decision-making and its disorders

Popular computational models of decision-making make specific assumptions about learning processes that may cause them to underfit observed behaviours. Here we suggest an alternative method using recurrent neural networks (RNNs) to generate a flexible family of models that have sufficient capacity to represent the complex learning and decision- making strategies used by humans. In this approach, an RNN is trained to predict the next action that a subject will take in a decision-making task and, in this way, learns to imitate the processes underlying subjects’ choices and their learning abilities. We demonstrate the benefits of this approach using a new dataset drawn from patients with either unipolar (n = 34) or bipolar (n = 33) depression and matched healthy controls (n = 34) making decisions on a two-armed bandit task. The results indicate that this new approach is better than baseline reinforcement-learning methods in terms of overall performance and its capacity to predict subjects’ choices. We show that the model can be interpreted using off-policy simulations and thereby provides a novel clustering of subjects’ learning processes—something that often eludes traditional approaches to modelling and behavioural analysis.Author summary: Computational models of decision-making provide a quantitative characterisation of the learning and choice processes behind human actions. Designing a computational model is often based on manual engineering with an iterative process to examine the consistency between different aspects of the model and the empirical data. In practice, however, inconsistencies between the model and observed behaviours can remain hidden behind examined summary statistics. To address this limitation, we developed a recurrent neural network (RNNs) as a flexible type of model that can automatically characterize human decision-making processes without requiring tweaking and engineering. To show the benefits of this new approach, we collected data on a decision-making task conducted on subjects with either bipolar or unipolar depression, as well as healthy controls. The results showed that, indeed, important aspects of decision-making remained uncaptured by typical computational models and even their enhanced variants, but were captured by RNNs automatically. Further, we were able to show that the nature of such processes can be unveiled by simulating the model under various conditions. This new approach can be used, therefore, as a standalone model of decision-making or as a baseline model to evaluate how well other candidate models fit observed data.