Theory: Embodied Musical Interaction

Theories of embodied interaction from the scientific field of human–computer interaction are discussed with an eye toward musical application.Music is a natural partner to human-computer interaction, offering tasks and use cases for novel forms of interaction. The richness of the relationship between a performer and their instrument in expressive musical performance can provide valuable insight to human-computer interaction (HCI) researchers interested in applying these forms of deep interaction to other fields. 

Despite the longstanding connection between music and HCI, it is not an automatic one, and its history arguably points to as many differences as it does overlaps. Music research and HCI research both encompass broad issues, and utilize a wide range of methods. I discuss how the concept of embodied interaction can be one way to think about music interaction. I propose how the three “paradigms” of HCI and three design accounts from the interaction design literature can serve as a lens through which to consider types of music HCI.  

The history of gestural musical instruments is presented, from the Theremin to instruments from the STEIM studio. I focus on the use of physiological signals to create music, from historical works of Lucier and Rosenboom to recent performances by myself and others. The body as musical instrument is discussed in a dynamic of coadaptation between performer and instrument in different configurations of body and technology.

The notion of the body as musical instrument explores the possibility of thinking of the human body as musical instrument. It builds on the philosophy of phenomenology to discuss body schemata that might be considered “instrumental” and discusses the diversity of bodies proposed by body theory to consider the incorporation of digital technology.

Reading list  

Laurel, B. (1991) Computers as theatre. Reading, Mass: Addison-Wesley Pub. 

Salter, C. and Sellars, P. (2010) Entangled: technology and the transformation of performance. Cambridge, Mass: MIT. 

Wang, G. (2018) Artful design: technology in search of the sublime. Stanford, CA: Standford University Press.