International Audio Laboratories Erlangen

Music Synchronization for RWC Music Database (Classical Music)

Introduction to Music Synchronization for RWC Music Database (Classical Music, RWC-MDB-C-2001)

On this web page, we supply MIDI-audio synchronization results obtained from the synchronization procedure described in [2]. Providing these results, our goal is to establish a platform for comparing and discussing various synchronization and alignment procedures. In general terms, music synchronization denotes a procedure which, for a given position in one representation of a piece of music, determines the corresponding position within another representation. Depending upon the respective data formats, one distinguishes between various synchronization tasks. For example, audio-audio synchronization refers to the task of time aligning two different audio recordings of a piece of music. These alignments can be used to jump freely between different interpretations, thus affording efficient and convenient audio browsing. The goal of MIDI-audio synchronization is to coordinate MIDI note events with audio data. The result can be regarded as an automated annotation of the audio recording with available MIDI data. For references, we refer to [1] [2] [3] [4].

To ensure the availability of necessary audio and MIDI data, we revert to the RWC (Real World Computing Music Database (RWC-MDB) . This database, which can be acquired by researchers at the RWC web page, is a copyright-cleared music database, see also [5]. The classical part of the RWC database contains an audio recording as well as a corresponding MIDI file for 61 pieces. Using the synchronization results based on [2], we modified each MIDI file to run synchronously to the corresponding audio recording. These modified MIDI files, which can be regarded as an annotation of the audio recordings by the MIDI note parameters, can now be obtained from this web page.

The Synchronization Results include the following data for each of the given 61 MIDI/audio pairs.

• Synchronized MIDI files (Warped MIDI [mid])
• MIDI annotations in form of synchronized MIDI event list (Annotation [csv])
• Audio-MIDI warping path (WP [csv])
• Sonification of first 60 seconds using a sinusoidal synthesis for the synchronized MIDI events (Sonification (sinusoidal) [mp3])
• Sonification of first 60 seconds using a wavetable synthesis for the synchronized MIDI events (Sonification (wavetable) [mp3])

Note: Because of copyright issues, this web page contains the synchronization results and sonifications only for the first 60 seconds of each recording. If you would like to obtain the full results, please obtain them from the AIST Annotation Web Page. If you have not obtained the RWC Music Database yet, you cannot use the data on this web page for your research without obtaining the RWC Music Database.

Disclaimer: The MP3 files and MIDI files supplied on this site can be used only for listening/evaluation purposes. Unauthorized reuse, reproduction, and redistribution are prohibited.

For comments and feedback, please contact Meinard Müller and Sebastian Ewert.

Synchronization Results

The table contains the following data.

• ID: Identifier for the piece similar to the RWC Piece No.
• Composer: Name of composer.
• Piece: Name and work number (opus etc.) of piece.
• Duration (Audio): Duration of audio recording in seconds.
• Duration (MIDI): Duration of MIDI version in seconds.
• Pitch Diff: Estimated difference between the audio and the MIDI given in semitones. In our synchronization approach, we account for possible tuning difference between the audio and the reference MIDI file by considering various pitch filter bank implementations. For example, the number 0.25 indicates that the audio tends to be 0.25 semitones above the reference tuning given by the MIDI file.
• MIDI Shift: Semitone shift applied to the MIDI file in the sonification to better match the audio. The MIDI shift number is the Pitch Diff number rounded to the next integer value.
• WP:Warping path resulting from the MIDI-audio synchronization procedure. The warping path is given as list of pairs, where each pair consists of a an audio time position and a MIDI time position both given in seconds. The resolution is 20 milliseconds using a feature rate of 50 Hz and classical DTW with the standard step size conditions.
• Annotation (Annot.): Text file containing a comma separated valued list of MIDI events along with corresponding time positions in the audio version. Each row contains the MIDI time position in seconds, the MIDI time position in musical beats, the MIDI note number, the MIDI note duration in seconds, the MIDI velocity, the MIDI channel, as well as the corresponding time position (computed by the synchronization procedure) in the audio recording.
• Warped MIDI: Temporally warped (and possibly pitch-corrected by MIDI Shift) MIDI running synchronously to the audio recording. Technically, we used tempo events in a first step to warp the MIDI on a beat level, and then, in a second step, minimally adjusted the tick times of the notes to allow for intra-beat micro corrections.
• Sonification (sinusoidal): MP3 file of sonification of the first 60 seconds of the audio using a sinusoidal synthesis for the synchronized MIDI events.
• Sonification (wavetable): MP3 file of sonification of the first 60 seconds of the audio using a wavetable synthesis for the synchronized MIDI events.

Note: The MIDI-audio alignment results were obtained using the state-of-the-art synchronization procedure described in [2]. No parameter tuning or manual postprocessing were applied to improve the final result. The results are not meant to be ground truth annotations and may contain synchronization errors and inaccuracies. In particular, boundary problems have not been dealt with in the current version so that synchronization inaccuracies may typically appear at the beginning and the end of the files.

Note: Because of copyright issues, this web page contains the synchronization results and sonifications only for the first 60 seconds of each recording. If you would like to obtain the full results, please obtain them from the AIST Annotation Web Page. If you have not obtained the RWC Music Database yet, you cannot use the data on this web page for your research without obtaining the RWC Music Database.

References

1. Meinard Müller
   Information Retrieval for Music and Motion
   Springer Verlag, ISBN: 3540740473, 2007. Bibtex Details

   @book{Mueller07_InformationRetrieval_SPRINGER,
   author = {Meinard M\"{u}ller},
   title = {Information Retrieval for Music and Motion},
   type = {Monograph},
   year = {2007},
   isbn = {3540740473},
   publisher = {Springer Verlag}
   url-details = {http://www.springer.com/computer/database+management+%26+information+retrieval/book/978-3-540-74047-6}
   }

2. Sebastian Ewert, Meinard Müller, and Peter Grosche
   High Resolution Audio Synchronization Using Chroma Onset Features
   In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP): 1869–1872, 2009. Bibtex PDF

   @inproceedings{EwertMG09_HighResAudioSync_ICASSP,
   author = {Sebastian Ewert and Meinard M{\"u}ller and Peter Grosche},
   title = {High Resolution Audio Synchronization Using Chroma Onset Features},
   booktitle = {Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing ({ICASSP})},
   address = {Taipei, Taiwan},
   month = apr,
   year = {2009},
   pages = {1869--1872},
   isbn = {978-1-4244-2354-5},
   issn = {1520-6149},
   url-pdf = {2009_EwertMuellerGrosche_HighResAudioSync_ICASSP.pdf}
   }

3. Meinard Müller, Henning Mattes, and Frank Kurth
   An Efficient Multiscale Approach to Audio Synchronization
   In Proceedings of the 7th International Conference on Music Information Retrieval (ISMIR): 192–197, 2006. Bibtex PDF

   @inproceedings{MuellerMK06_EfficientMultiscaleApproach_ISMIR,
   author = {Meinard M{\"u}ller and Henning Mattes and Frank Kurth},
   title = {An Efficient Multiscale Approach to Audio Synchronization},
   booktitle = {Proceedings of the 7th International Conference on Music Information Retrieval ({ISMIR})},
   address = {Victoria, Canada},
   month = oct,
   year = {2006},
   pages = {192--197},
   url-pdf = {2006_MuellerMattesKurth_MultiscaleAudioSynchronization_ISMIR.pdf}
   }

4. Meinard Müller, Frank Kurth, and Tido Röder
   Towards an Efficient Algorithm for Automatic Score-to-Audio Synchronization
   In Proceedings of the 5th International Conference on Music Information Retrieval (ISMIR): 365–372, 2004. Bibtex PDF

   @inproceedings{MuellerKR04_AutomaticScoreAudioSync_ISMIR,
   author = {Meinard M{\"u}ller and Frank Kurth and Tido R{\"o}der},
   title = {Towards an Efficient Algorithm for Automatic Score-to-Audio Synchronization},
   booktitle = {Proceedings of the 5th International Conference on Music Information Retrieval ({ISMIR})},
   address = {Barcelona, Spain},
   month = oct,
   year = {2004},
   pages = {365--372},
   url-pdf = {2004_MuellerKurthRoeder_ScoreAudioSynchronization_ISMIR.pdf}
   }

5. Masataka Goto, Hiroki Hashiguchi, Takuichi Nishimura, and Ryuichi Oka
   RWC Music Database: Popular, Classical and Jazz Music Databases
   In Proceedings of the International Conference on Music Information Retrieval (ISMIR), 2002. Bibtex PDF

   @inproceedings{GotoHHNO02_ISMIR,
   author = {Masataka Goto and Hiroki Hashiguchi and Takuichi Nishimura and Ryuichi Oka},
   title = {{RWC} Music Database: Popular, Classical and Jazz Music Databases},
   booktitle = {Proceedings of the International Conference on Music Information Retrieval (ISMIR)},
   address = {Paris, France},
   year = {2002},
   url-pdf = {2002_GotoHNO_RWC_ISMIR.pdf}
   }
   ### Links
   * []()

Links

• Prof. Dr. Meinard Müller, International Audio Laboratories Erlangen

• Prof. Dr. Michael Clausen, Multimedia Signal Processing Group, Bonn University

• Dipl.-Inform. Sebastian Ewert, Multimedia Signal Processing Group, Bonn University

• Prof. Dr. Masataka Goto, Media Interaction Group, National Institute of Advanced Industrial Science and Technology (AIST)