|
Solving Delayed Coordination Problems in MAS Host Publication: Proceedings of the Adaptive and Learning Agents Workshop 2011 Authors: Y. De Hauwere, P. Vrancx and A. Nowé Publication Date: May. 2011 Number of Pages: 7
Abstract: Recent research has demonstrated that considering local interactions
among agents in specific parts of the state space, is a successful way
of simplifying the multi-agent learning process. By taking into account
other agents only when a conflict is possible, an agent can
significantly reduce the state-action space in which it learns.
Current approaches, however, consider only the immediate rewards for
detecting conflicts. This restriction is not suitable for realistic
systems, where rewards can be delayed and often conflicts between agents
become apparent only several time-steps after an action has been taken.
In this paper, we contribute a reinforcement learning algorithm that
learns where a strategic interaction among agents is needed, several
time-steps before the conflict is reflected by the (immediate) reward
signal. To do this, we make use of statistical information about the
future returns and the state information of the agents. This allows the
agent to determine when it should expand its state representation with
information on the other agents and when it can safely rely on its own
state information. We apply our method to a set of representative grid
world problems and show that with our approach, agents successfully
manage to expand their state information to solve delayed coordination
problems.
|
