Asymmetric multiagent reinforcement learning

Article type: Research Article

Authors: Könönen, Ville

Affiliations: Neural Networks Research Centre, Helsinki University of Technology, P.O. Box 5400, FI-02015 HUT, Finland. Tel.: +358 9 451 5024; Fax: +358 9 451 3277; E-mail: [email protected]

Abstract: A novel model for asymmetric multiagent reinforcement learning is introduced in this paper. The model addresses the problem where the information states of the agents involved in the learning task are not equal; some agents (leaders) have information how their opponents (followers) will select their actions and based on this information leaders encourage followers to select actions that lead to improved payoffs for the leaders. This kind of configuration arises e.g. in semi-centralized multiagent systems with an external global utility associated to the system. We present a brief literature survey of multiagent reinforcement learning based on Markov games and then propose an asymmetric learning model that utilizes the theory of Markov games. Additionally, we construct a practical learning method based on the proposed learning model and study its convergence properties. Finally, we test our model with a simple example problem and a larger two-layer pricing application.

Keywords: multiagent reinforcement learning, Markov games, Nash equilibrium, Stackelberg equilibrium, convergence

Journal: Web Intelligence and Agent Systems: An international journal, vol. 2, no. 2, pp. 105-121, 2004