Learning Macroeconomic Policies based on Microfoundations: A Dynamic Stackelberg Mean Field Game Approach

The Lucas critique emphasizes the importance of considering the impact of policy changes on the expectations of micro-level agents in macroeconomic policymaking. However, the inherently self-interested nature of large-scale micro-agents, who pursue long-term benefits, complicates the formulation of optimal macroeconomic policies. This paper proposes a novel general framework named Dynamic Stackelberg Mean Field Games (Dynamic SMFG) to model such policymaking within sequential decision-making processes, with the government as the leader and households as dynamic followers. Dynamic SMFGs capture the dynamic interactions among large-scale households and their response to macroeconomic policy changes. To solve dynamic SMFGs, we propose the Stackelberg Mean Field Reinforcement Learning (SMFRL) algorithm, which leverages the population distribution of followers to represent high-dimensional joint state and action spaces. In experiments, our method surpasses macroeconomic policies in the real world, existing AI-based and economic methods. It allows the leader to approach the social optimum with the highest performance, while large-scale followers converge toward their best response to the leader's policy. Besides, we demonstrate that our approach retains effectiveness even when some households do not adopt the SMFG policy. In summary, this paper contributes to the field of AI for economics by offering an effective tool for modeling and solving macroeconomic policy-making issues.