Exploration strategies in n-Person general-sum multiagent reinforcement learning with sequential action selection

Article type: Research Article

Authors: Akramizadeh, Ali^{; *} | Afshar, Ahmad | Menhaj, Mohammad B.

Affiliations: Department of Electrical Engineering, Amir Kabir University of Technology, Tehran, Iran

Correspondence: [*] Corresponding author. E-mail: [email protected].

Abstract: In this paper, two novel exploration strategies are proposed for n-person general-sum multiagent reinforcement learning with sequential action selection. The existing learning process, called extensive Markov game, is considered as a set of successive extensive form games with perfect information. We introduce an estimated value for taking actions in games with respect to other agents' preferences which is called associative Q-value. They can be used to select actions probabilistically according to Boltzmann distribution. Simulation results present the effectiveness of the proposed exploration strategies that are used in our previously introduced extensive-Q learning methods. Regarding the complexity of existing methods of computing Nash equilibrium points, if it is possible to assume sequential action selection among agents, extensive-Q will be more convenient for dynamic task multiagent systems with more than two agents.

Keywords: Multiagent reinforcement learning, Nash equilibrium points, exploration-exploitation tradeoff, Markov games

DOI: 10.3233/IDA-2011-0502

Journal: Intelligent Data Analysis, vol. 15, no. 6, pp. 913-929, 2011