Body Movement Generation For Expressive Violin Performance Applying Neural Networks

Generating body movements based on given music audio recordings is an emerging research topic. This problem remains challenging particularly for string instruments, considering the fact that the relationship between the musical note sequences and the body movement sequences in string instruments does not have an one-to-one correspondence and is highly context-dependent. In this paper, we take a divide-and-rule approach to tackle the multifaceted characteristics of musical movement, and propose a framework for generating violinists' body movements. Both objective and subjective evaluations show that the proposed framework improves the stability as well as the perceptual quality of the generation outputs by using the task-specific models for bowing and expressive movement. To the best of our knowledge, this work represents the first attempt to generate violinists' body movements considering music expression.