International Audio Laboratories Erlangen

A Separate and Restore Approach to Score-Informed Music Decomposition

This is the accompanying website for the paper "A Separate and Restore Approach to Score-Informed Music Decomposition" by Christian Dittmar, Jonathan Driedger, and Meinard Müller.

Abstract

Our goal is to improve the perceptual quality of signal components extracted in the context of music source separation. Specifically, we focus on decomposing polyphonic, mono-timbral piano recordings into the sound events that correspond to the individual notes of the underlying composition. Our separation technique is based on score-informed Non-Negative Matrix Factorization (NMF) that has been proposed in earlier works [2,3,4] as an effective means to enforce a musically meaningful decomposition of piano music. However, the method still has certain shortcomings for complex mixtures where the tones strongly overlap in frequency and time. As the main contribution of this paper, we propose a restoration stage based on refined Wiener filter masks to score-informed NMF. Our idea is to introduce notewise soft masks created from a dictionary of perfectly isolated piano tones, which are then adapted to match the timbre of the target components. On this page, we present some separation results from our test set.

We used the physical modeling plugin "Pianoteq" to synthesize interpreted MIDI files from the "Saarland Music Database". All MIDI notes in the excerpts were split into a bass (left hand) and a treble set (right hand). For further details about the items in the test set, please refer to [1]. Further down this page we provide selected audio examples, that are marked in the following diagram of evaluation results:

Specifically, example 1 represents the test item with the best OPS score for the conventional method. Example 2 represents the test item, where both the conventional approach as well as the proposed restore approach yield the worst results. Example 3 represents a test item, where the proposed method yields the highest OPS. Using the radio buttons in the player plugin below, you can mute and unmute the bass tones and treble tones, respectively. Alternatively, you can download the corresponding WAV-files from the table below.

Audio Examples

Example 1: Bach_BWV849-02_001_20090916-SMD (SMD number: 2) [Excerpt from 0:12.518 to 0:28.893]

Example 2: Chopin_Op010-03_007_20100611-SMD (SMD number: 22) [Excerpt from 0:0.000 to 0:25.000]

Example 3: Ravel_JeuxDEau_008_20110315-SMD (SMD number: 48) [Excerpt from 0:9.587 to 0:25.409]

References

1. Christian Dittmar, Jonathan Driedger, and Meinard Müller
   A Separate and Restore Approach to Score-Informed Music Decomposition
   In Proceedings of IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA), 2015. Bibtex

   @inproceedings{Dittmar:2015:WASPAA,
   author    = {Christian Dittmar and Jonathan Driedger and Meinard M{\"u}ller},
   title     = {A Separate and Restore Approach to Score-Informed Music Decomposition},
   booktitle = {Proceedings of {IEEE} Workshop on Applications of Signal Processing to Audio and Acoustics ({WASPAA})},
   address   = {New Paltz, New York, USA},
   year      = {2015},
   }

2. Sebastian Ewert, Bryan Pardo, Meinard Müller, and Mark Plumbley
   Score-Informed Source Separation for Musical Audio Recordings
   IEEE Signal Processing Magazine, 31(3): 116–124, 2014. Bibtex

   @article{Ewert:2014:IEEE-SPS,
   author    = {Sebastian Ewert and Bryan Pardo and Meinard M{\"u}ller and Mark Plumbley},
   title     = {Score-Informed Source Separation for Musical Audio Recordings},
   journal   = {IEEE Signal Processing Magazine},
   volume    = {31},
   number    = {3},
   year      = {2014},
   month     = {April},
   pages     = {116--124}
   }

3. Jonathan Driedger, Harald Grohganz, Thomas Prätzlich, Sebastian Ewert, and Meinard Müller
   Score-Informed Audio Decomposition and Applications
   In Proceedings of the ACM International Conference on Multimedia (ACM-MM): 541–544, 2013. Bibtex

   @inproceedings{Driedger:2013:ACMMM,
   author    = {Jonathan Driedger and Harald Grohganz and Thomas Pr{\"a}tzlich and Sebastian Ewert and Meinard M{\"u}ller},
   title     = {Score-Informed Audio Decomposition and Applications},
   booktitle = {Proceedings of the ACM International Conference on Multimedia ({ACM-MM})},
   address   = {Barcelona, Spain},
   year      = {2013},
   pages     = {541--544},
   }

4. Sebastian Ewert and Meinard Müller
   Using score-informed constraints for NMF-based source separation
   In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 129–132, 2012. Bibtex

   @inproceedings{ewert:2012:icassp,
   Author = {Sebastian Ewert and Meinard M{\"u}ller},
   Booktitle = {Proceedings of the {IEEE} International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
   Pages = {129--132},
   Title = {Using score-informed constraints for {NMF}-based source separation},
   year = {2012},
   address = {Kobe, Japan}
   }