License: Creative Commons Attribution 3.0 Unported license (CC BY 3.0)
When quoting this document, please refer to the following
DOI: 10.4230/DFU.Vol3.11041.1
URN: urn:nbn:de:0030-drops-34637
URL: http://dagstuhl.sunsite.rwth-aachen.de/volltexte/2012/3463/
Go to the corresponding DFU Volume Portal

Thomas, Verena ; Fremerey, Christian ; Müller, Meinard ; Clausen, Michael

Linking Sheet Music and Audio - Challenges and New Approaches

pdf-format:
2.pdf (4 MB)

Abstract

Score and audio files are the two most important ways to represent,
convey, record, store, and experience music. While score describes a piece of music on an abstract level using symbols such as notes, keys, and measures, audio files allow for reproducing a specific acoustic realization of the piece. Each of these representations reflects different facets of music yielding insights into aspects ranging from structural elements (e.g., motives, themes, musical form) to specific performance aspects (e.g., artistic shaping,
sound). Therefore, the simultaneous access to score and audio
representations is of great importance.

In this paper, we address the problem of automatically generating
musically relevant linking structures between the various data sources
that are available for a given piece of music. In particular, we discuss the task of sheet music-audio synchronization with the aim to link regions in images of scanned scores to musically corresponding sections in an audio recording of the same piece. Such linking structures form the basis for novel interfaces that allow users to access and explore multimodal sources of music within a single framework.

As our main contributions, we give an overview of the state-of-the-art for this kind of synchronization task, we present some novel approaches, and indicate future research directions. In particular, we address problems that arise in the presence of structural differences and discuss challenges when applying optical music recognition to complex orchestral scores. Finally, potential applications of the synchronization results are presented.

BibTeX - Entry

@InCollection{thomas_et_al:DFU:2012:3463,
  author =	{Verena Thomas and Christian Fremerey and Meinard M{\"u}ller and Michael Clausen},
  title =	{{Linking Sheet Music and Audio - Challenges and New Approaches}},
  booktitle =	{Multimodal Music Processing},
  pages =	{1--22},
  series =	{Dagstuhl Follow-Ups},
  ISBN =	{978-3-939897-37-8},
  ISSN =	{1868-8977},
  year =	{2012},
  volume =	{3},
  editor =	{Meinard M{\"u}ller and Masataka Goto and Markus Schedl},
  publisher =	{Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik},
  address =	{Dagstuhl, Germany},
  URL =		{http://drops.dagstuhl.de/opus/volltexte/2012/3463},
  URN =		{urn:nbn:de:0030-drops-34637},
  doi =		{10.4230/DFU.Vol3.11041.1},
  annote =	{Keywords: Music signals, audio, sheet music, music synchronization, alignment, optical music recognition, user interfaces, multimodality}
}

Keywords: Music signals, audio, sheet music, music synchronization, alignment, optical music recognition, user interfaces, multimodality
Collection: Multimodal Music Processing
Issue Date: 2012
Date of publication: 27.04.2012

DROPS-Home | Fulltext Search | Imprint | Privacy Published by LZI