Learning Algorithms for Cooperative Multi-Agent Systems

Decision making in distributed systems requires coordination among a number of individually acting agents, in order to achieve a common goal. Therefore, the agents' behavior policies must be harmonized perfectly. The notion 'Multi Agent Reinforcement Learning' subsumes learning techniques, which enable agents to acquire cooperative decision policies automatically - based on a specification of the system's desired overall behavior. Application fields for distributed agents include resource management, scheduling, network routing, or robotics.

The capabilities of learning systems - as compared to non-adaptive solutions - for real-world tasks have been exemplified by numerous researchers. Most of the learning approaches published are of empirical nature and, in many cases, originate from an adaptation of learning algorithms that were designed for the single-agent case originally. Although guarantees for the validity and optimality of the behavior strategies found with those approaches cannot be given in general, the solutions found empirically are often better than existing standard solutions. As a result of lacking theretical grounding, however, many of the concepts applied cannot be generalized.

Based on our group's existing work in the context of Multi Agent Reinforcement Learning we are investigating the learning in distributed systems both empirically and theoretically within the scope of this project. Main goals include the development of distributed learning algorithms with verifiable convergence properties for stochastic environments, the analysis of modelling variants as well as their behavior regarding convergence, robustness and generalization of the solutions found.

The project is being conducted in the years 2005 to 2009 within the Neuroinformatics Group at the University of Osnabrück under the direction of Prof. Dr. Martin Riedmiller and Dr. Thomas Gabel. The project is funded by the DFG (Deutsche Forschungsgemeinschaft).

• T. Gabel. Cooperative Multi-Agent Systems from the Reinforcement Learning Perspective - Challenges, Algorithms, and an Application. Dagstuhl Seminar Proceedings 09371, Algorithmic Methods for Distributed Cooperative Systems. LZI, Dagstuhl, Germany, 2009. pdf
• T. Gabel and M. Riedmiller. Reinforcement Learning for DEC-MDPs with Changing Action Sets and Partially Ordered Dependencies. Proceedings of the 7th International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS 2008). Springer, Estoril, Portugal, 2008. short full
• T. Gabel and M. Riedmiller. On a Successful Application of Multi-Agent Reinforcement Learning to Operations Research Benchmarks. Proceedings of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL 07). Honolulu, USA, 2007. pdf
• T. Gabel and M. Riedmiller. Multi-Agent Case-Based Reasoning for Cooperative Reinforcement Learners. In Proceedings of the 8th European Conference on Case-Based Reasoning (ECCBR 2006). Fethiye, Turkey, 2006. pdf
• T. Gabel and M. Riedmiller. Reducing Policy Degradation in Neuro-Dynamic Programming. In Proceedings of the 11th European Symposium on Artificial Neural Networks (ESANN 2006). Bruges, Belgium, 2006. pdf

Researchers working in our team:

• Prof. Dr. Martin Riedmiller
• Dr. Thomas Gabel

For more information on this research project, please contact Martin Riedmiller.