International Audio Laboratories Erlangen

Ferienakademie 2024, Sarntal (22.09. - 04.10.2024)

Course 8: Learning with Music Signals

Main Tutor/Lecturer: Simeon Rau, Prof. Dr. Michael Sedlmair

Group: Data Visualization for Music Representations

Music visualization involves creating visual representations of music that serve a variety of applications. It can be used to analyze personal music listening histories or assist both musicologists and non-experts in storing, structuring, and analyzing vast amounts of musical data, facilitating tasks like structure identification and modulation detection. Music visualization may also be used for instrument practice, interactive AI-assisted composition, and augmented reality tools that overlay responsive visualizations. In this group, we discuss various visualization approaches that bridge the gap between complex manual analysis and fully automated methods, providing scalable, multi-faceted insights and enhancing the overall musical experience.

Literature

  1. Richard Khulusi, Jakob Kusnick, Christofer Meinecke, Christina Gillmann, Josef Focht, and Stefan Jänicke
    A Survey on Visualizations for Musical Data
    Computer Graphics Forum, 39(6): 82–110, 2020. PDF DOI 
    @article{KhulusiKMGFJ20_SurveyVisMusic_CGF,
author       = {Richard Khulusi and Jakob Kusnick and Christofer Meinecke and Christina Gillmann and Josef Focht and Stefan J{\"{a}}nicke},
title        = {A Survey on Visualizations for Musical Data},
journal      = {Computer Graphics Forum},
volume       = {39},
number       = {6},
pages        = {82--110},
year         = {2020},
url          = {https://doi.org/10.1111/cgf.13905},
doi          = {10.1111/CGF.13905},
url-pdf   = {2020_KhulusiEtAl_VisMusicDataComputerSurvey_CGF.pdf}
}
  2. Matthias Miller, Daniel Fürst, Hanna Hauptmann, Daniel A. Keim, and Mennatallah El-Assady
    Augmenting Digital Sheet Music through Visual Analytics
    Computer Graphics Forum, 41(1): 301–316, 2022. PDF DOI 
    @article{MillerFHKE22_SheetMusicVis_CGF,
author       = {Matthias Miller and Daniel F{\"{u}}rst and Hanna Hauptmann and Daniel A. Keim and Mennatallah El{-}Assady},
title        = {Augmenting Digital Sheet Music through Visual Analytics},
journal      = {Computer Graphics Forum},
volume       = {41},
number       = {1},
pages        = {301--316},
year         = {2022},
url          = {https://doi.org/10.1111/cgf.14436},
doi          = {10.1111/CGF.14436},
url-pdf   = {2022_MillerFHKE_VisSheetMusicAnalytics_CGF.pdf}
}
  3. Matthias Miller, Julius Rauscher, Daniel A. Keim, and Mennatallah El-Assady
    CorpusVis: Visual Analysis of Digital Sheet Music Collections
    Computer Graphics Forum, 41(3): 283–294, 2022. PDF DOI 
    @article{MillerRKE22_CorpusVis_CGF,
author       = {Matthias Miller and Julius Rauscher and Daniel A. Keim and Mennatallah El{-}Assady},
title        = {{CorpusVis}: {V}isual Analysis of Digital Sheet Music Collections},
journal      = {Computer Graphics Forum},
volume       = {41},
number       = {3},
pages        = {283--294},
year         = {2022},
url          = {https://doi.org/10.1111/cgf.14540},
doi          = {10.1111/CGF.14540},
url-pdf   = {2022_MillerRKE_VisCorpusSheetMusic_EuroVis-CGF.pdf}
}
  4. Simeon Rau, Frank Heyen, Stefan Wagner, and Michael Sedlmair
    Visualization for AI-Assisted Composing
    In Proceedings of the International Society for Music Information Retrieval Conference (ISMIR): 151–159, 2022. PDF 
    @inproceedings{RauH0S22_VisComposingAI_ISMIR,
author       = {Simeon Rau and Frank Heyen and Stefan Wagner and Michael Sedlmair},
title        = {Visualization for AI-Assisted Composing},
booktitle    = {Proceedings of the International Society for Music Information
Retrieval Conference ({ISMIR})},
pages        = {151--159},
year         = {2022},
url          = {https://archives.ismir.net/ismir2022/paper/000017.pdf},
url-pdf   = {2022_RauHWS_VisComposingAI_ISMIR.pdf}
}
  5. Frank Heyen, Quynh Quang Ngo, and Michael Sedlmair
    Visual Overviews for Sheet Music Structure
    In Proceedings of the International Society for Music Information Retrieval Conference, (ISMIR): 692–699, 2023. PDF DOI 
    @inproceedings{HeyenNS23_VisOverviewSheetMusicStructure_ISMIR,
author       = {Frank Heyen and Quynh Quang Ngo and Michael Sedlmair},
title        = {Visual Overviews for Sheet Music Structure},
booktitle    = {Proceedings of the International Society for Music Information
Retrieval Conference, ({ISMIR})},
pages        = {692--699},
year         = {2023},
url          = {https://doi.org/10.5281/zenodo.10265383},
doi          = {10.5281/ZENODO.10265383},
url-pdf   = {2023_HeyenNS_VisSheetMusicStructure_ISMIR.pdf}
}
  6. Lima, Hugo B., Santos, Carlos G. R. Dos, and Meiguins, Bianchi S.
    A Survey of Music Visualization Techniques
    ACM Comput. Surv., 54(7), 2021. PDF DOI 
    @article{Lima2021_MusicVisSurvey_ACM,
author = {Lima, Hugo B. and Santos, Carlos G. R. Dos and Meiguins, Bianchi S.},
title = {A Survey of Music Visualization Techniques},
year = {2021},
address = {New York, NY, USA},
volume = {54},
number = {7},
url = {https://doi.org/10.1145/3461835},
doi = {10.1145/3461835},
journal = {ACM Comput. Surv.},
url-pdf = {2021_LimaEtAl_MusicVisSurvey_ACM.pdf}
}
  7. Chang, Baofeng, Sun, Guodao, Li, Tong, Huang, Houchao, and Liang, Ronghua
    MUSE: Visual Analysis of Musical Semantic Sequence
    IEEE Transactions on Visualization and Computer Graphics, 29(9): 4015–4030, 2023. PDF DOI 
    @article{Chang2023_MUSE_TVCG,
author={Chang, Baofeng and Sun, Guodao and Li, Tong and Huang, Houchao and Liang, Ronghua},
journal={IEEE Transactions on Visualization and Computer Graphics},
title={MUSE: Visual Analysis of Musical Semantic Sequence},
year={2023},
volume={29},
number={9},
pages={4015-4030},
doi={10.1109/TVCG.2022.3175364},
url-pdf={2023_ChangEtAl_MUSE_TVCG.pdf}}

Further Links