Agent Based Model Discovery with Reinforcement Learning

Abstract

Agent Based Modelling (ABM) is a powerful tool for modelling social systems. Generative runs simulate micro-level behaviours that give rise to emergent macro-level outcomes. To ensure the accuracy of those outcomes to the modelled process, behavioural rules are carefully implemented and their parameters calibrated. Recently, methods for the inverse generation of ABMs - from outcomes to behavioural rules - have received much attention. Most approaches aim at constructing parts of the ABM or require high-resolution data. In this thesis, we use Reinforcement Learning (RL) to learn the individual policies of a school choice model using only summary statistics of the reference process. A Deep Q-Network is used to learn and encode the recovered policy, which can then be used in simulations. We demonstrate the robustness of our method for the recovery of different latent behavioural rules using different reward functions. We find that our method is not very robust, although it shows signs of learning. In subsequent experiments, we show that the recovered policies generalise better than a baseline random agent, but the learned behaviour only partially matches the reference. We speculate on two critical obstacles to the performance that future research should address.