Lecture: Selected Topics in Deep Learning for Audio, Speech, and Music Processing (Winter Term 2026/27)
- Instructors: Dr.-Ing. Stefan Balke, Dr.-Ing. Andreas Brendel, Dr. Ben Maman, Prof. Dr. Meinard Müller
- Credits: 2.5 ECTS
- Time (Lecture): Winter Term 2026/27, Tuesday, 16:00–17:45 (1. Lecture: 13.10.2026)
- Exam (graded): Written examination at the end of term
- Dates (Lecture)): To be announced
- Examination Dates (Room 3R4.03): To be announced
Content
Many recent advances in audio, speech, and music processing have been driven by deep learning (DL), leading to major improvements in tasks such as source separation, audio synthesis, analysis, and music transcription. This course focuses on selected DL-based approaches for audio signals, with emphasis on modern generative models (including diffusion and flow matching), dataset curation, and differentiable signal processing (e.g., DDSP). Further topics include diffusion-based music synthesis, nonnegative autoencoders for audio decomposition, and differentiable alignment methods. These examples illustrate how complex structures and relationships can be learned from data while addressing challenges in realistic scenarios.
Rather than providing a comprehensive overview, the course focuses on generally applicable methods and offers a critical perspective on their potential and limitations. A central objective is the integration of domain knowledge into neural network architectures to obtain more interpretable and robust models. Based on recent research literature, the material emphasizes conceptual clarity and mathematical depth, and concludes with practical aspects such as implementation, reproducibility, and dataset handling.
Course Requirements
In this course, we require a good knowledge of deep learning techniques, machine learning, and pattern recognition as well as a strong mathematical background. Furthermore, we require a solid background in general digital signal processing and some experience with audio, image, or video processing.
It is recommended to finish the following modules (or having equivalent knowledge) before starting this module:
Links
Examination
There will be a written exam (60 minutes). Further details will be given in the lecture.
Lecture: Topics, Material, Instructions
The course consists of two overview-like lectures, where we introduce current research problems in audio, speech, and music processing. We will then continue with 6 to 8 lectures wich are based on articles from the research literature. The lecture material includes handouts of slides, links to the original articles, and possibly links to demonstrators and further online resources. In the following list, you find links to the material. If you have any questions regarding the lecture, please contact the listed instructors.