Design for virtuosity: How expert musical skills can be repurposed in the design of new instruments.
Musical performers spend many years achieving proficiency on their instruments. Newly created digital musical instruments (DMIs) face a significant barrier to adoption in that few performers are willing to repeat these years of training to develop expertise on an unknown instrument. Without expert players, evaluating the success of a DMI design is challenging, and establishing its place in a broader musical community is nearly impossible. As a result, while many digital instruments have been created over the past decade, few have achieved lasting impact beyond the first few performances.
The Design for Virtuosity project aims to model and repurpose the existing expertise of trained musicians, so that new creative possibilities are offered without requiring thousands of hours of retraining.
Understanding skills through performer-instrument models
A first focus of the project concerns the understanding of the interaction between performer and instrument leading to the development of models. Instrumental performance can be considered a special case of human-machine interaction which is interesting both for its complexity and for the common experience that the musical instrument becomes an extension of the body: while playing, the performer is often not consciously thinking about the instrument, particularly in a virtuosic performance. Controlled experiments and participatory design exercises will establish how an instrument’s design affects the development of expertise, and how existing expertise can be transferred to newly-created instruments.
Designing new DMIs informed by the performer-instrument models
Secondly, the resulting models will inform the DMI creation process, taking a holistic approach unifying hardware design, digital signal processing, human-computer interaction (HCI) and artistic considerations. Existing DMIs often implicitly prioritise the convenience of the computer over the experience of the human player. On acoustic instruments, the entire physical object contributes to the sound, however subtly, but the choice of sensors in a DMI typically reduces the performer’s actions to just a few machine-tractable dimensions. This fellowship will create instruments which deliberately oversample the interaction, using more sensors and higher sampling rates than apparently necessary, not to create a more complicated instrument, but rather to capture the subtle nuances that experts prize. Evaluation of the new DMIs will help refine the original models of performer-instrument interaction.
References
F. Morreale, J. Morreale and A. McPherson. Effect of Instrument Structure Alterations on Violin Performance. Frontiers in Psychology 9 (2436), 2018.
K. Buys and A. McPherson. Real-time bowed string feature extraction for performance applications. Proc. Sound and Music Computing, Limassol, Cyprus, 2018.
