International Audio Laboratories Erlangen

Learning with Music Signals: Technology Meets Education (LEARN)

Logo_DFG Teaser_LEARN_small Logo_FAU

The core mission of the LEARN project is to approach and explore the concept of learning from different angles using music as a challenging and instructive application domain. The project is funded by the German Research Foundation as part of the Reinhart Koselleck Programme. On this website, we summarize the project's main objectives and provide links to project-related resources (data, demonstrators, websites) and publications.

Project Description

Learning with Music Signals: Technology Meets Education

Teaser_LEARN_large

The revolution in music distribution, storage, and consumption has fueled tremendous interest in developing techniques and tools for organizing, analyzing, retrieving, and presenting music-related data. As a result, the field of music information retrieval (MIR) has matured over the last 20 years into an independent research area related to many different disciplines, including signal processing, machine learning, information retrieval, musicology, and the digital humanities. This project aims to break new ground in technology and education in these disciplines using music as a challenging and instructive multimedia domain. The project is unique in its way of approaching and exploring the concept of learning from different angles. First, learning from data, we will build on and advance recent deep learning (DL) techniques for extracting complex features and hidden relationships directly from raw music signals. Second, by learning from the experience of traditional engineering approaches, our objective is to understand better existing and to develop more interpretable DL-based systems by integrating prior knowledge in various ways. In particular, as a novel strategy with great potential, we want to transform classical model-based MIR approaches into differentiable multilayer networks, which can then be blended with DL-based techniques to form explainable hybrid models that are less vulnerable to data biases and confounding factors. Third, in collaboration with domain experts, we will consider specialized music corpora to gain a deeper understanding of both the music data and our models' behavior while exploring the potential of computational models for musicological research. Fourth, we will examine how music may serve as a motivating vehicle to make learning in technical disciplines such as signal processing or machine learning an interactive pursuit. Through our holistic approach to learning, we want to achieve significant advances in the development of explainable hybrid models and reshape how recent technology is applied and communicated in interdisciplinary research and education.

Projektbeschreibung

Teaser_LEARN_large

Lernen mit Musiksignalen: Technologie trifft Ausbildung

Die erheblichen Fortschritte in der Art, wie wir Musik verbreiten, speichern und nutzen, haben ein großes Interesse an der Entwicklung von Techniken und Werkzeugen zum Organisieren, Analysieren, Abrufen, Suchen und Präsentieren musikbezogener Daten hervorgerufen. Infolgedessen hat sich das Gebiet des Music Information Retrieval (MIR) in den letzten 20 Jahren zu einem eigenständigen Forschungsgebiet mit Bezug zu ganz unterschiedlichen Disziplinen wie der Signalverarbeitung, dem Maschinellen Lernen, des Information Retrieval, den Musikwissenschaften und den Digital Humanities entwickelt. Dieses Projekt hat zum Ziel, neue Wege in der Technologieentwicklung und Ausbildung in diesen Disziplinen zu beschreiten, wobei die Musik als herausfordernde und instruktive Domäne multimedialer Daten dient. Die Einzigartigkeit des Projekts besteht darin, dass wir uns dem Konzept des Lernens aus verschiedenen Blickwinkeln annähern. Erstens werden wir neuartige Techniken des maschinellen Lernens, basierend auf Deep Learning (DL), erforschen, um komplexe Merkmale und verborgene Beziehungen direkt aus den Musiksignalen zu extrahieren. Zweitens besteht unser Ziel darin, aus den Erfahrungen traditioneller Ingenieursansätze zu lernen, um zum einen bestehende DL-basierte Systeme besser zu verstehen und zum anderen durch Integration von Vorwissen interpretierbarere Systeme zu entwickeln. Als neuartige Strategie mit großem Potenzial wollen wir insbesondere klassische modellbasierte MIR-Ansätze in differenzierbare Multilayer-Netzwerke überführen. Diese sollen dann mit DL-basierten Techniken zu erklärbaren Hybridmodellen, die weniger anfällig für Ungleichgewichte in den Daten (data bias) und Störfaktoren (confounding factors) sind, fusioniert werden. Drittens werden wir in Zusammenarbeit mit Domänenexperten spezialisierte Musikkorpora betrachten, um ein tieferes Verständnis sowohl der Musikdaten als auch des Verhaltens unserer Modelle zu erlangen und gleichzeitig das Potenzial von computerbasierten Methoden für die musikwissenschaftliche Forschung zu untersuchen. Viertens soll uns die Musik als motivierendes Medium dienen, um das Lernen in technischen Disziplinen wie der Signalverarbeitung oder dem maschinellen Lernen interaktiv zu gestalten. Durch unseren ganzheitlichen Ansatz des Lernens wollen wir nicht nur erhebliche Fortschritte bei der Entwicklung erklärbarer Hybridmodelle erzielen, sondern auch die Anwendung und Vermittlung neuer Technologien in interdisziplinärer Forschung und Lehre von Grund auf umgestalten.

Projected-Related Activities

Projected-Related Resources and Demonstrators

The following list provides an overview of the most important publicly accessible sources created in the LEARN project:

Projected-Related Publications

The following publications reflect the main scientific contributions of the work carried out in the LEARN project.

  1. Peter Meier, Ching-Yu Chiu, and Meinard Müller
    A Real-Time Beat Tracking System with Zero Latency and Enhanced Controllability
    Transaction of the International Society for Music Information Retrieval (TISMIR), 7(1): 213–227, 2024. PDF DOI 
    @article{MeierCM24_RealTimePLP_TISMIR,
author = {Peter Meier and Ching-Yu Chiu and Meinard M{\"u}ller},
title = {A Real-Time Beat Tracking System with Zero Latency and Enhanced Controllability},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {7},
number = {1},
pages = {213--227},
year = {2024},
doi = {10.5334/tismir.189},
url-pdf   = {2024_MeierCM_RealTimePLP_TISMIR_ePrint.pdf},
}
  2. Johannes Zeitler, Christof Weiß, Vlora Arifi-Müller, and Meinard Müller
    BPSD: A Coherent Multi-Version Dataset for Analyzing the First Movements of Beethoven's Piano Sonatas
    Transaction of the International Society for Music Information Retrieval (TISMIR), 7(1): 195–212, 2024. PDF Demo DOI 
    @article{ZeitlerWAM24_BPSD_TISMIR,
author = {Johannes Zeitler and Christof Wei{\ss} and Vlora Arifi-M{\"u}ller and Meinard M{\"u}ller},
title = {{BPSD}: {A} Coherent Multi-Version Dataset for Analyzing the First Movements of {B}eethoven's Piano Sonatas},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {7},
number = {1},
pages = {195--212},
year = {2024},
doi = {10.5334/tismir.196},
url-pdf   = {2024_ZeitlerWAM24_BPSD_TISMIR_ePrint.pdf},
url-demo = {https://zenodo.org/records/12783403}
}
  3. Meinard Müller and Ching-Yu Chiu
    A Basic Tutorial on Novelty and Activation Functions for Music Signal Processing
    Transaction of the International Society for Music Information Retrieval (TISMIR), 7(1): 179–194, 2024. PDF Demo DOI 
    @article{MuellerC24_TutorialNovelty_TISMIR,
author = {Meinard M{\"u}ller and Ching-Yu Chiu},
title = {A Basic Tutorial on Novelty and Activation Functions for Music Signal Processing},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {7},
number = {1},
pages = {179--194},
year = {2024},
doi = {10.5334/tismir.202},
url-pdf   = {2024_Mueller_TutorialNovelty_TISMIR_ePrint.pdf},
url-demo = {https://github.com/groupmm/edu_novfct}
}
  4. Meinard Müller, Simon Dixon, Anja Volk, Bob L. T. Sturm, Preeti Rao, and Mark Gotham
    Introducing the TISMIR Education Track: What, Why, How?
    Transaction of the International Society for Music Information Retrieval (TISMIR), 7(1): 85–98, 2024. PDF Details DOI 
    @article{MuellerDVSRG24_RealTimePLP_TISMIR,
author = {Meinard M{\"u}ller and Simon Dixon and Anja Volk and Bob L. T. Sturm and Preeti Rao and Mark Gotham},
title = {Introducing the {TISMIR} Education Track: {W}hat, Why, How?},
journal = {Transaction of the International Society for Music Information Retrieval ({TISMIR})},
volume = {7},
number = {1},
pages = {85--98},
year = {2024},
doi = {10.5334/tismir.199},
url-details = {https://transactions.ismir.net/articles/10.5334/tismir.199},
url-pdf   = {2024_MuellerEtAl_EditorialEducation_TISMIR_ePrint.pdf}
}
  5. Yigitcan Özer, Leo Brütting, Simon Schwär, and Meinard Müller
    libsoni: A Python Toolbox for Sonifying Music Annotations and Feature Representations
    Journal of Open Source Software (JOSS), 9(96): 1–6, 2024. PDF Demo DOI 
    @article{OezerBSM24_SonificationToolbox_JOSS,
author    = {Yigitcan {\"O}zer and Leo Br{\"u}tting and Simon Schw{\"a}r and Meinard M{\"u}ller},
title     = {libsoni: {A} {P}ython Toolbox for Sonifying Music Annotations and Feature Representations},
journal   = {Journal of Open Source Software ({JOSS})},
volume    = {9},
number    = {96},
year      = {2024},
pages     = {1--6},
doi       = {10.21105/joss.06524},
url-demo  = {https://github.com/groupmm/libsoni},
url-pdf   = {2024_OezerBSM_SonificationToolbox_JOSS_ePrint.pdf}
}

Projected-Related Ph.D. Theses

    Links