Deep Learning
Dr. J. Boedecker, J. T. Springenberg

• HISinOne entry (Praktikum)
• Announcements:
  • first meeting: 21.10.2015

• Dates
  • Wednesday, 10:00 - 11:30am, building 082 - SR 00 029

• Language
  • English

The lab course on deep learning architectures is intended to get familiar with a recent paradigm in machine learning, the so-called deep learning architectures. They are based on neural networks and overcome the problems of classical learning algorithms for neural networks by combining supervised and unsupervised techniques. During recent years amazing results have been achieved with these techniques. Application areas can be found mainly in pattern recognition.

The lab course will start with a short seminar phase in which the participants read and discuss research papers on the topic of deep learning architectures to get a basic idea of this technique. After that, we will apply deep learning techniques to pattern recognition tasks in computer vision and data analysis.

• Presentation: Introduction to deep learning (pdf) (videos playable in full-screen)
• Presentation: Implementing simple MLPs (pdf) (videos playable in full-screen)
• Presentation: Implementing CNNs, the presentation I gave was based on Andrej's great slides on implementing CNNs from here and his course notes can be found here
• Presentation: Otions for the last exercises (pdf)

• You can find the course assignments on github feel free to fork them (oh and pull requests for improvements will be considered ;))

• Draft of a recent book on deep learning by Yoshua Bengio, Ian Goodfellow and Aaron Courville (link)
• Course on Neural Networks by Hugo Larochelle (link)
• Deep learning tutorial by Kyunghyun Cho (pdf)
• Stanford course on deep learning and unsupervised feature learning (link)
• A great recent course by Andrej Karpathy on convolutional neural networks (link)
• NEW The smiley data can be found here (without labels) (link) should you want to use label information (you don't have to but) here is a version with labels: (link)
• NEW A good starting network for CIFAR-10 can be found here: https://github.com/Lasagne/Recipes/blob/master/modelzoo/cifar10_nin.py . Look here: https://github.com/Lasagne/Recipes/blob/master/examples/Using%20a%20Caffe%20Pretrained%20Network%20-%20CIFAR10.ipynb for an ipython notebook were that network is used. For loading the data simply use the python arrays provided on the CIFAR-10 website.