CLSquare: Closed Loop Simulation System

CLS² (pronounced: clsquare) provides a standardized framework for testing (not only) Reinforcement Learning controllers on a (growing) number of different plants.

Features:

• Easy combination of different plants and controllers by definition in a configuration file.
• Lean interfaces for plants and controllers for a quick migration to and from CLS² from and to other software environments.
• Optional graphical interface and visualization.
• Many demos for easy starting.

Current plants:

• Simulations
  • Cart Pole (single, double and parallel), underactuated pole
  • Maze
  • Mountain Car
• Hardware Interfaces
  • Kinova Jaco
  • Robotis Bioloid

Current controllers:

• Various static standard controllers (linear, PID…)
• Table-based Q Reinforcement Learning
• Neural Q and policy controllers
• Gaussian Processes
• Kernel-based batch controller (Ormoneit and Sen, 2002)
• Fitted Q Iteration with Extra Trees (Ernst, Geurts and Wehenkel, 2005)

The software is developed and tested under:

• Ubuntu Linux 12.04
• Mac OSX 10.8

Basic requirements:

• a C++ compiler (build-essential for Ubuntu, xcode for OSX)
• CMake 2.6.3 (2.8.2 is required for libgp)

Each module that has special requirements will report them during installation if they are missing.

The latest software release can be downloaded from here.

1. Get a fresh copy of CLS²
2. Unpack the source code
   • tar xzf clsquare.tgz
3. Compile the program
   • mkdir build
   • cd build
   • cmake ..
   • make install

The most recent manual is included in the source package. For convenience reasons, you can find the manual here, too: CLSquare Manual v4.0

In addition, documentation for the available modules can be generated using Doxygen by typing make doc in the program's build folder.

Researchers contributing to this project:

• Prof. Dr. Martin Riedmiller
• Manuel Blum
• Thomas Lampe
• Dr. Roland Hafner
• Dr. Sascha Lange
• Dr. Stephan Timmer

For more information on this research project, please contact Martin Riedmiller.