Mobile manipulation in natural environments -- "TidyUpRobot"

The project Mobile manipulation in natural environments – “TidyUpRobot” was part of a larger project involving several groups at University of Freiburg in the context of the Willow Garage PR2-Beta program. As one of 11 selected projects worldwide, its goal was to foster the state-of-the-art in personal mobile robotics.

In the course of this project, our group worked on learning algorithms for efficient grasping skills for use on real-world robot platforms, as a basis for more complex behaviors in household environments. To this end, we developed highly data-efficient Reinforcement Learning algorithms, which are able to learn reactive feedback controllers for systems with unknown dynamics and real-time constraints in a very short time (both in terms of interactions time, and time to learn a successful behavior policy). Furthermore, we presented new approaches for reactive behaviors for grasping moving objects, which are capable of integrating constraints with respect to obstacles into the learning process. We also extended a method for real-time object tracking to enable tracking of multiple objects simultaneously.

All of the software which was developed as part of the project has been developed as modules in open source frameworks. Many of these have been publicly released, or are planned to be released in the future.

Available software packages developed within the project

Packages for the Robot Operating System (ROS Fuerte Turtle):

Multi-Object Tracking-Learning-Detection

Model Predictive Control

Reactive Grasping

The modules for the improved MPC-based reactive grasping with learned dynamic models (AGP-ILQR), as well as the ones based on neural fitted Q-iteration (NFQ) (visual servoing and approximate model-assisted NFQ) were written for CLSquare and are planned to be made available in a future update.

Publications

2014

Thomas Lampe, Martin Riedmiller (2014) Approximate Model-Assisted Neural Fitted Q-Iteration. In IEEE International Joint Conference on Neural Networks (IJCNN 2014). Beijing, China. PDF Bibtex

@inproceedings{LampeRiedmillerIJCNN2014,
  author = {Thomas Lampe and Martin Riedmiller},
  title = {Approximate Model-Assisted Neural Fitted Q-Iteration},
  booktitle = {IEEE International Joint Conference on Neural Networks (IJCNN 2014)},
  address = {Beijing, China},
  year = {2014},
  keywords = {NFQ;Model-based Learning;EXC1086;BLBT;Neurobots;PR2},
  file = {lampe_riedmiller_ijcnn14.pdf}
}

Joschka Boedecker, Jost Tobias Springenberg, Jan Wülfing, Martin Riedmiller (2014) Approximate Real-Time Optimal Control Based on Sparse Gaussian Process Models. In Adaptive Dynamic Programming and Reinforcement Learning (ADPRL). PDF Bibtex

@INPROCEEDINGS{BoeSprWuel2014,
  author = {Joschka Boedecker and Jost Tobias Springenberg and Jan Wülfing and Martin Riedmiller},
  title = {Approximate Real-Time Optimal Control Based on Sparse Gaussian Process Models},
  booktitle = {Adaptive Dynamic Programming and Reinforcement Learning (ADPRL)},
  year = {2014},
  file = {adprl-2014.pdf},
  keywords= {EXC1086;BLBT;Neurobots;PR2;BoedeckerSelect}
}

2013

Thomas Lampe, Martin Riedmiller (2013) Acquiring Visual Servoing Reaching and Grasping Skills using Neural Reinforcement Learning. In IEEE International Joint Conference on Neural Networks (IJCNN 2013). Dallas, TX. PDF Bibtex

@inproceedings{LampeRiedmiller2013,
  author = {Thomas Lampe and Martin Riedmiller},
  title = {Acquiring Visual Servoing Reaching and Grasping Skills using Neural Reinforcement Learning},
  booktitle = {IEEE International Joint Conference on Neural Networks (IJCNN 2013)},
  address = {Dallas, TX},
  year = {2013},
  file = {lampe2013.pdf},
  keywords = {NFQ;Grasping;Visual Servoing;EXC1086;BLBT;Neurobots;PR2}
}