This is an archived version of the Machine Learning Lab website for documentation purposes. The host is in no way affiliated with the University of Freiburg, its Faculty of Engineering or any of its members.

Open Source Initiative CLSquare

CLSquare: Closed Loop Simulation System

CLS2 (pronounced: clsquare) provides a standardized framework for testing (not only) Reinforcement Learning controllers on a (growing) number of different plants.

Features:

  • Easy combination of different plants and controllers by definition in a configuration file.
  • Lean interfaces for plants and controllers for a quick migration to and from CLS2 from and to other software environments.
  • Optional graphical interface and visualization.
  • Many demos for easy starting.

Screenshot of the cart pole plant

Current plants:

  • Simulations
    • Cart Pole (single, double and parallel), underactuated pole
    • Maze
    • Mountain Car
  • Hardware Interfaces

Current controllers:

  • Various static standard controllers (linear, PID…)
  • Table-based Q Reinforcement Learning
  • Neural Q and policy controllers
  • Gaussian Processes
  • Kernel-based batch controller (Ormoneit and Sen, 2002)
  • Fitted Q Iteration with Extra Trees (Ernst, Geurts and Wehenkel, 2005)

Supported Platforms

The software is developed and tested under:

  • Ubuntu Linux 12.04
  • Mac OSX 10.8

Basic requirements:

  • a C++ compiler (build-essential for Ubuntu, xcode for OSX)
  • CMake 2.6.3 (2.8.2 is required for libgp)

Each module that has special requirements will report them during installation if they are missing.

Download

The latest software release can be downloaded from here.

Installation

  1. Get a fresh copy of CLS2
  2. Unpack the source code
    • tar xzf clsquare.tgz
  3. Compile the program
    • mkdir build
    • cd build
    • cmake ..
    • make install

Documentation

The most recent manual is included in the source package. For convenience reasons, you can find the manual here, too: CLSquare Manual v4.0

In addition, documentation for the available modules can be generated using Doxygen by typing make doc in the program's build folder.

People

Researchers contributing to this project:

  • Prof. Dr. Martin Riedmiller
  • Manuel Blum
  • Thomas Lampe
  • Dr. Roland Hafner
  • Dr. Sascha Lange
  • Dr. Stephan Timmer

Contact

For more information on this research project, please contact Martin Riedmiller.