Sequence Learning

Elective for CS grad students at the University of Applied Sciences Rosenheim.

Class Schedule and Credits

Time and Location: Tuesdays at 11.45, B0.08a

Comunication via Mattermost (register).

Format: Each week, we will discuss algorithms and their theory before implementing them to get a better hands-on understanding. Java is suggested, pairprogramming encouraged, BYOD strongly recommended!

Credits: oral exam (15’) at the end of the semester, tentative date: July 10 (last day of classes).

Note: Materials will be in English, the lectures/tutorials will be taught in German; the oral exam in the language of your choice.

  • Niemann, H: Klassifikation von Mustern. 2. Überarbeitete Auflage, 2003 (available online)
  • Huang, Acero, Hon: Spoken Language Processing: A Guide to Theory, Algorithm and System Development. (ISBN-13: 978-0130226167)
  • Goodfellow, I and Bengio,Y and Courville, A: Deep Learning. 2016 (available online)
  • Jurafsky, D and Martin, J: Speech and Language Processing. 2017 (available online)
  • Manning, C, Raghavan P and Schütze, H: Introduction to Information Retrieval, Cambridge University Press. 2008. (available online)


  • March 20: Introduction. (slides, exercise)

    We’ll start with the general concepts of supervised vs. unsupervised learning and classification of independent observations vs. sequences of observations. To get you motivated, we’ll look at a list of recent “AI products” that utilize sequence learning.

  • March 27: Auto-Correct. (slides by Ben Langmead, exercise)

    We’ll start with a classic implementation of auto-correcting mispelled words to bring dynamic programming back to memory. We’ll also look at scalability regarding computation and memory efforts.

  • April 3: Easter holidays

  • April 10: no class (extra classes on 4/24 and 5/8)

  • April 17: States and Cost Functions. (slides, exercise)

    Understand how DP can be used on an abstraction of distances and states. We’ll build a smarter, keyboard layout aware auto-correct and start looking into some applications in signal processing (isolated word and DTMF sequence classification).

  • April 24: Modeling Sequences. (slides, exercise)

    Learn about n-grams, a simple yet effective approach to learn contexts of distcrete symbols. We’ll use n-grams to improve our auto-correct by incorporating context and suggesting following words.

  • May 1: Labor Day

  • Maz 8: Hidden Markov Models. (slides, exercise)

    We’ll take a close look at hidden Markov models and how to (efficiently) evaluate and train them. The Viterbi decoding algorithm tells us the most likely sequence and the path that lead to it. We’ll use them to build a proof-of-concept isolated word recognizer.

  • May 15: Higher-Level Sequence Modeling with HMM. (slides, exercise)

    Learn how to model complex sequences of arbitrary length that prohibit explicit modeling, such as speech recognition or choreographies in sports. Here we will combine what we’ve discussed so far: prefix trees, n-gram models and efficient search.

  • May 22: Pentecost

  • May 29: Feed-Forward Neural Networks (extended class, 9.45-3.15). (slides perceptron and nnets,, exercise).

    A brief introduction to neural networks: fundamentals, topologies and training. We’ll skip implementing the details and use tensorflow for the examples. Did you know that you could program fizzbuzz as a neural network? Please have Python with Numpy and TensorFlow installed and operational on your machine!

  • June 5: no class (see extended classes on 5/29 and 6/12)

  • June 12: Recurrent Neural Networks (extended class, 11.45-5.00). (slides cs231n: RNNs, exercise)

    Recurrent neural networks use feedback loops to introduce temporal context or “memory” into the network. We’ll study them using two examples: language modeling and drawing classification.

  • June 19: Sequence to Sequence Learning. (literature and exercise, attention slides, CCL’17 tutorial slides)

    Previous algorithms explicitly modeled the sequence, either as a graph-like structure such as an HMM or by concatenating observations to a single data point. Encoder-decoder networks are a special kind of topology of recurrent neural networks that can be used to model sequence to sequence mappings, such as found in end-to-end speech recognition, machine translation or automatic summarization – without explicitly modeling states!

  • June 26: Deep Learning: Practical Considerations. (slides: toolkits, practical considerations, deployment, exercise)

    We’ll compare different deep learning toolkits and their requirements or potential to get a grip on what’s necessary to apply them to a new problem.

  • July 3: Review, Q&A and Exam Prep

    We’ll recap the topics we’ve covered and work through a set of example questions to prepare for the oral exam. Please come prepared to get the most out of this class!

  • July 10: oral exams

Subscribe to repository to follow updates.