International Audio Laboratories Erlangen

Adaptive Pitch-Shifting with Applications to Intonation Adjustment in A Cappella Recordings

This is the accompanying website to the article [1]. Our publicly available Python toolbox is based on the Matlab implementation in [2].

1. Sebastian Rosenzweig, Simon Schwär, Jonathan Driedger, and Meinard Müller
   Adaptive Pitch-Shifting with Applications to Intonation Adjustment in A Cappella Recordings
   In Proceedings of the International Conference on Digital Audio Effects (DAFx): 121–128, 2021. Bibtex Details

   @inproceedings{RosenzweigSDM21_AdaptivePitchShifting_DAFX,
   title       = {Adaptive Pitch-Shifting with Applications to Intonation Adjustment in A Cappella Recordings},
   author      = {Sebastian Rosenzweig and Simon Schw{\"a}r and Jonathan Driedger and Meinard M{\"u}ller},
   booktitle   = {Proceedings of the International Conference on Digital Audio Effects ({DAFx})},
   year        = {2021},
   pages       = {121--128},
   address     = {Vienna, Austria},
   url-details = {http://production.audiolabs.uni-erlangen.de/resources/MIR/2021-DAFX-AdaptivePitchShifting}
   }

2. Jonathan Driedger and Meinard Müller
   TSM Toolbox: MATLAB Implementations of Time-Scale Modification Algorithms
   In Proceedings of the International Conference on Digital Audio Effects (DAFx): 249–256, 2014. Bibtex Details

   @inproceedings{DriedgerM14_TSM_DAFx,
   author    = {Jonathan Driedger and Meinard M{\"u}ller},
   title     = {{TSM} {T}oolbox: {MATLAB} Implementations of Time-Scale Modification Algorithms},
   booktitle = {Proceedings of the International Conference on Digital Audio Effects ({DAFx})},
   address   = {Erlangen, Germany},
   year      = {2014},
   pages     = {249--256},
   url-details = {https://www.audiolabs-erlangen.de/resources/MIR/TSMtoolbox}
   }

Abstract

A central challenge for a cappella singers is to adjust their intonation and to stay in tune relative to their fellow singers. During editing of a cappella recordings, one may want to adjust local intonation of individual singers or account for global intonation drifts over time. This requires applying a time-varying pitch-shift to the audio recording, which we refer to as adaptive pitch-shifting. In this context, existing (semi-)automatic approaches are either labor intensive or face technical and musical limitations. In this work, we present automatic methods and tools for adaptive pitch-shifting with applications to intonation adjustment in a cappella recordings. To this end, we show how to incorporate time-varying information into existing pitch-shifting algorithms that are based on resampling and time-scale modification (TSM). Furthermore, we release an open-source Python toolbox, which includes a variety of TSM algorithms and an implementation of our method. Finally, we show the potential of our tools by two case studies on global and local intonation adjustment in a cappella recordings using a publicly available multitrack dataset of amateur choral singing.

libtsm

If you publish results obtained using libtsm, please cite [1].

libtsm (Python)
Matlab TSM Toolbox
Demo Notebook

Audio Examples

Global Intonation Adjustment	Local Intonation Adjustment

Excerpt (as shown in the figures)

Full performance

Acknowledgments

This project is supported by the German Research Foundation (DFG MU 2686/12-1, MU 2686/13-1). The International Audio Laboratories Erlangen are a joint institution of the Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU) and Fraunhofer Institute for Integrated Circuits IIS. We thank Edgar Suarez, El Mehdi Lemnaouar and Miguel Gonzales for implementation support and Sascha Disch for valuable comments.