Advanced Machine Learning
Prof. Dr. Martin Riedmiller, Jost Tobias Springenberg, Jan Wülfing

• Dates:
  • Weekly meetings: 12:00-16:00 in room 082-00-028 (Pool)

• Credit points:
  • 6 ECTS (can be different depending on the course of studies)
• Language:
  • Englisch

This lab course is a unique blend of lecture, online course material, seminar and practical, following the idea of research inspired teaching. It is intended to make you familiar with a new paradigm in machine learning, the so-called deep learning architectures. They are based on neural networks and overcome the problems of classical learning algorithms for neural networks by combining supervised and unsupervised techniques. During recent years amazing results have been achieved with these techniques. The lab course will mainly focus on practical aspects and efficient implementations, but will also entail studying scientific publications on artificial neural networks.

• Presentation: Introduction to deep learning (pdf) (videos playable in full-screen)
• Presentation: A short review of feed-forward neural networks (pdf) (videos playable in full-screen)
• Presentation: A short introduction to Torch (pdf)
  • The corresponding code examples can be found here (link)
• NEW Presentation: Assignment for phase four (pdf)

• Draft of a recent book on deep learning by Yoshua Bengio, Ian Goodfellow and Aaron Courville (link)
• Course on Neural Networks by Hugo Larochelle (link)
• Deep learning tutorial by Kyunghyun Cho (pdf)
• Stanford course on deep learning and unsupervised feature learning (link)
• Deep learning code tutorials in theano (pdf)
• Online Demos
  • Imagenet demo from the Toronto group (link)
  • Clarifain demo (link)
• Software
  • We recommend using either python (with numpy), lua (with torch) or julia / matlab (with the builtin matrix capabilities) for the first exercise
• Datasets
  • The MNIST dataset in original format can be found here (link)
  • A numpy version of MNIST can be found here: (link) (note that this version has the training data already split up into training and validation set)
  • A torch version of loading MNIST can be found here: (link)
  • A script for loading the data in python can be found (link)
  • Functions for loading the data in torch/lua can be found here: (link)
• Phase2
  • Torch7 can be found here (link)
  • A Torch7 cheat-sheet can be found here (link)
  • Basic Demos: a bunch of demos/tutorials to get started (link)
  • Deep-Learning Tutorials: supervised and unsupervised learning (link)
  • NEW You can download the script which you should adapt to make predictions on unknown data (here)
  • NEW A set of test digits is available here (here) (Note: I have kept these at 32×32 pixels so that those of you who worked with padded images have an easier job, if you need 28×28 pixel images just crop the center from the examples)
  • NEW the full test-dataset, including the labels can now be found here if you want to verify your results
• Phase3
  • NEW the smiley dataset, training examples and test images ZIP. Note that the data are stored in binary mode, i.e. load them without the 'ascii' option.
  • NEW the STL-10 dataset can be found (here)

• Group 1: Farooq, Anam
• Group 2: Johannes, Anton
• Group 3: Muneeb, Mohammad
• Group 4: Tulio, Ecem, Stefan

• Group 1: Anam, Mohammad
  • Paper: Best Practicies for Convlutional Neural Networks Aplied to Visual Document Analysis
• Group 2: Ecem, Anton
  • Paper: Improving neural networks by preventing co-adaptation of feature detectors
• Group 3: Tulio, Muneeb
  • Paper: ImageNet Classification with Deep Convolutional Neural Networks
• Group 4: Farooq, Stefan
  • Paper: An Analysis of Single-Layer Networks in Unsupervised Feature Learning

• Group 1: Anam, Ecem
• Group 2: Mohammad, Anton
• Group 3: Tulio, Farooq
• Group 4: Muneeb, Stefan