Learning the IPA market with individual and social rewards

Article type: Research Article

Authors: Gomes, Eduardo Rodrigues | Kowalczyk, Ryszard

Affiliations: Faculty of Information and Communication Technologies, Swinburne University of Technology, John Street, Hawthorn, VIC 3122, Australia

Note: [] Corresponding author. E-mail: [email protected].

Abstract: Market-based mechanisms offer a promising approach for distributed resource allocation. In this paper we consider the Iterative Price Adjustment, a pricing mechanism that can be used in commodity-market resource allocation systems. We address the scenario where agents use utility functions to describe preferences in the allocation and learn demand functions optimized for the market by Reinforcement Learning. In particular, we investigate reward functions based on the individual utilities of the agents and the Social Welfare of the market. We also evaluate the quality of demand functions obtained throughout the learning process with the aim of analyzing its influence on the behavior of the agents and exploring how much learning is enough, so the amount required can be reduced. This investigation shows that both reward functions deliver similar results when the market consists of only learning agents. We further investigate this behavior and present its theoretical-experimental explanation.

Keywords: Market-based resource allocation, Iterative Price Adjustment, Reinforcement Learning, multiagent systems

DOI: 10.3233/WIA-2009-0158

Journal: Web Intelligence and Agent Systems: An International Journal, vol. 7, no. 2, pp. 123-138, 2009