1. Relationality

Sound is intrinsically and unignorably relational: it emanates, propagates, communicates, vibrates, and agitates; it leaves a body and enters others; it binds and unhinges, harmonizes and traumatises; it sends the body moving, the mind dreaming, the air oscillating. It seemingly eludes definition, while having profound effect.
— Brandon LaBelle^[1]

The relational character of sound is a key concern in this PhD project, guiding the development of a novel ultrasonic installation technology, the practice of composing sonic artworks with it and the design of qualitative research studies exploring listener experience. Sonic experience is examined as a relational phenomenon, mediated by specific design qualities of the artwork, social and material agencies unique to the context in which it is presented. Relationality also becomes an epistemological issue; methodologies come to be recognised as discursive and material apparatuses^[2] that actively shape the collection and interpretation of data. In response, efforts are taken to adopt epistemic positions that are grounded in the situated and contingent experience of listening.

In a further relational move, the disciplines of sound art and HCI are set into dialogue, interfered and diffracted by this PhD. Resonances, dissonances, and more pertinently, the effects of difference^[3], are discerned as an emergent outcome of interdisciplinary perspective. Indeed, key contributions include provocations for HCI to recognise facets of interactive experience that become illuminated through an orientation to the sonic and challenge disciplinary conventions.

2. Interviews with Situated Sound Art Practitioners

The project began with an interview study conducted with 10 sound artists and composers, all engaged in situated sonic practices. The interviews probed the creative process and explored how a sound artist’s methods and tools might influence the reception of their work. A thematic analysis^[4] of interview transcriptions led to a characterisation of artist processes as mediatory, in the sense that they act in between site and audience experience and are guided by the non-human agencies of settings and material things. Artists appear to transfer their own situated and embodied listening to that of the audience and develop sonic and staging devices to direct perceptual activity and listening attention. The findings also highlighted a number of engagement challenges, in particular the difficulty artists face in understanding their audience’s experience and the specificity of an artwork’s effect not just to its location but also to the disposition, abilities and prior experiences of listeners.

Read more: 'On Mediating Space, Sound and Experience: Interviews with situated sound art practitioners'

3. Ultrasonic Art Installation

Informed by sound art methods and the interview study just described, a novel ultrasonic technology was developed that harnesses listener entanglement with sound as the source of interactive effects. Two sound installation artworks were created using this technology - Being With The Waves (2021), Sonographies (2024) - and employed as research probes in three subsequent participant studies. Inaudible to the naked ear, the artwork is heard via custom headphones and, as an experience, manifests differently for individual listeners according to their spatial orientation to speakers, body position, movement, and perceptual behaviour. There are no movement sensors, programmed interactivity, mappings or feedback loops. Instead, the experience of listening may be perceived as interactive due to the spatial arrangement of sound sources and the close entanglement of the listener with sound phenomena physically present in the air. The interactive experience emerges as an embodied sensory experience determined by the listener’s orientation to sounds, physical movement, and perceptual behaviour.

The technology design was guided by the following intentions:  
— To preserve the acoustic entanglement of body, sound, and space as the origin of interactivity, rather than programmed event-loops and mappings.
— To augment this paradigm using digital technology as a mediating layer that transforms the appearance of sound.
— To emphasise the spatiality of sound and its physical presence in the installation space.
— To encourage close and attentive listening together with an active awareness of the body's relationship to sound and its physicality.

Technical Implementation

Eight individual tweeters form a spatial arrangement in the exhibition space. A multi-channel composition is composed in Ableton in the audible range. Each audio track is shifted into the ultrasonic range using the technique of amplitude modulation, simply multiplying the audio signal by a sine tone of 20.5kHz using a Max4Live device before it is output to an individual tweeter in the installation space. Thus, the installation physically sounds in the exhibition space but above the range of human hearing. Visitors hear the artwork via custom headphones. Microphones on the outside of the earcups capture ultrasonic phenomena at the ear, and a microcontroller shifts the signals down into audibility via the same process of amplitude modulation before outputting them to headphones.

The audio fidelity of the original multi-channel composition remains remarkably intact, but ultrasonic modulation skews the spatial appearance of sounds and results in exaggerated Doppler effects: listeners hear the audio warp and shudder in a manner that is hypersensitive and intimately connected to their bodily behaviour. There are no movement sensors, mappings or feedback loops. Instead, the experience of listening may be perceived as interactive due to the spatial arrangement of sound sources and the close entanglement of the listener with sound phenomena physically present in the air.

For further technical details, see:
‍'Being With The Waves: An ultrasonic art installation enabling rich interaction without sensors'

Being With The Waves

The first artwork created with the ultrasonic installation technology is called 'Being With The Waves'. Having listened to and experimented with a variety of different sound materials through the technology, I sought to establish a kind of sonic language; ways of composing sound for and with the ultrasonic technology that would also reinforce experiential qualities of interaction.

Being With The Waves was conceived in response to a call for artworks to be included in the Ramsgate Festival of Sound, where the festival theme for 2021 was to be 'Stories from the Sea'. In response, the installation includes excerpts from micro-phenomenological interviews^[5] conducted with Ramsgate residents exploring sensory experiences in and with the sea - how it feels to catch a wave, float, or be submersed - and these are interwoven with an immersive musical composition of drones, instrumental textures and field recordings. The installation, therefore, through the felt quality of movement-sound interaction, is intended to elicit for the listener an experience of fluidity and of being with the phenomena of sound. Through the field recordings of waves and boats clanking, the installation also signifies the themes of sea, fluidity and bodily experience.

4. The Social Mediation of Sound Installation Listening

The first participant study explored physical behaviours associated with sound installation listening and their mediation by social factors. An ethnomethodological^[6] study was conducted during the public exhibition of Being With The Waves at Ramsgate Festival of Sound in 2021. Arguably, the installation is antisocial by design, foregrounding private rather than collective experience. However, up to six listeners may experience it together, creating a social dimension that inevitably mediates behaviour. An interaction analysis^[7] of video observations identified important ways in which the physical behaviour of listeners appears to be mediated by the presence (or absence) of other people in the exhibition space. The study’s findings indicate that the local social dynamic strongly impacts how listeners move and use their bodies, which affects the discovery and exploration of spatial and interactive effects.

Read more: 'Listening Together-Apart: On the social mediation of sound installation listening'

5. Exploring the Felt Quality of Sonic Interaction

The second participant study employed the micro-phenomenological interview technique^[5] to help listeners describe particular moments within their experience of Being With The Waves in fine detail. Individual participants experienced the installation at a university performance space before hour-long interviews. These produced breadth of experiential descriptions, including auditory perceptions, intentions, bodily behaviour, mental imagery, emotional states, thought processes, and semantic interpretations.

The data analysis phase began on my return from a period of maternity leave. I began reading the data and considering the analysis approach in the context of my broader PhD project and its aims: What does it mean to be working with sound? What kind of knowledge does the installation design itself imply? How might my epistemological choices extend from that?

‍I found Karen Barad's philosophy of agential realism^[8] resonated with the contingent and situated character of listening, and perceived interactivity in the installation and sought an analysis approach that would align epistemologically. I chose to conduct a diffractive analysis^[9] to examine listener experiences of entanglement with sound in the installation. Materialist theories of sound were `plugged-in'^[10] to the interview data producing an emergent series of readings, which I conceive as patterns of positive interference, where theory and experiential descriptions resonate. This led to a clustering of phenomenological descriptions around particular theories:
‍
— Reading the data with Voegelin/ Merleau-Ponty’s notion of ‘being-honeyed'^[11] [Voegelin, 2010, p. 9] surfaced data describing the felt quality of connection with sound as material.
— Plugging the mobile and temporal quality of sonic material^[12] into the data, led to the identification of temporal instability of embodiment sensations.
— Szendy's proposal that listeners 'punctuate' sound with their listening^[13] surfaces descriptions of intention setting with bodily poses and movement qualities.
— Plugging the doubtfulness and ambiguity of listening^[14] into the data surfaces ways in which listener expectations regarding the system design lead listeners to dwell in technological 'dis-illusions'.

Additionally, thinking with sound, I became aware of emergent differences concerning other accounts of entangled experience in HCI and taken-for-granted, invisible dynamics of research methods. These wider reverberations informed a series of provocations for HCI to embrace qualities of the sonic and consider epistemological positions grounded in other sense modalities, for example to:

— Design for an entangled subjectivity
— Sense less, know less, quantify less
— Assume dynamism

‍Read more:
‍ 'Thinking With Sound: Exploring the experience of listening to an ultrasonic art installation'

6. Examining the Situated Design and Gallery Curation of Sonographies

The final research study of the Phd explored the entangled activities of composing and curating a new sound installation with the ultrasonic technology called Sonographies at no format Gallery in South London.

Sonographies

Where Being With The Waves foregrounded a representational mode of understanding - apprehending the meaning of texts and sounds as signifiers of maritime themes, Sonographies is intended to emphasise an embodied mode of understanding, using abstract or ambiguous sound design that suggests (rather than overtly signifies) themes of the domestic, intimate and personal space.

Sonographies is entirely composed of sound materials collected at my home. Mirroring the way that the installation appears to magnify phenomena, sounds were recorded with a sensitive microphone in close proximity to reveal intricate detail that would otherwise be barely audible. The motoric whirr of a small clock mechanism, the background hum of household appliances, rustle of duvet covers and breath of my sleeping son were recorded in this way. Resonant frequencies of the gallery space were translated to musical pitches and recorded as sustained chords on the piano in the living room. Again, the microphones were placed in close proximity to the strings and captured the sound of piano hammers, the beating and dancing of harmonics.

The extent to which the installation is perceived to be interactive depends on the sound materials from which it is composed and their mediation by the installation's ultrasonic technologies. The presence and audibility of the Doppler Effect depend entirely on sound design choices. With the sound design of Sonographies, I sought to frame interactive effects more prominently in the artwork and organise the composition around different flavours of body-sound relationship.

The Study

This study centred on my own artistic practice, creating Sonographies during a creative residency at no format Gallery in South London. Inspired by the site-specific approaches of sound artists, I worked to create links between the sonic composition and acoustic features of the gallery, attempting a more-than-human disposition whereby nonhuman material influences are invited into the process and allowed to shape artistic practice. I adopted research through design (RtD) methods to examine my process, presenting my findings as a design journey^[16]: a reflective first-person account of the design process constructed from contemporaneous notes and recorded conversations with my curatorial collaborator Eva Martinez.
‍
I found that attuning to the materiality of the gallery led to feelings of discomfort and doubt, potentially due to an underlying tension between the installation’s listening technology and site-specific practices. Upgrades to the technology^[14] emphasised the strong mediatory effects of the technology on the listening experience, and only when my orientation shifted to working with these effects did my aesthetic intentions begin to cohere with and become amplified by the ultrasonic technology.

This study also highlighted the importance of curation in shaping the experience of listeners. Brief post-experience surveys suggest that the intention to foreground embodied modes of experience was successful, with many listeners referring to sensations of closeness, intimacy and hyper-awareness of the body. Adopting a relational stance in the analysis of survey responses highlighted unexpected social sound-making, multiple interpretations of the installation and the contingent, situated character of responses.

Read more: 'Negotiating Entanglements in the Composition and Curation of an Ultrasonic Art Installation'

Conclusions

In conclusion, this research explores the subtle entanglements of listener(s), sound, technology and place, showing that interactive experience can emerge not only through computational systems, but art assemblages that harness the materiality of sound and the relationality of listening. By bringing sound art into dialogue with human–computer interaction, and by centring the experiences of listening audiences, I have proposed ways of designing and researching that recognise the agency of people, materials, and environments alike. The installations and studies presented in this thesis are offered as both technical and methodological implementations, and as provocations for HCI researchers to think with sound and listen closely to the phenomena under study. Listening is never passive, but an active participation in the making of the world; a search, not for fixed answers, but for shifting and contingent patterns of understanding. Such contingency, I contend, is a vital orientation for design, one that acknowledges the specificities of context, sharpens sensitivity to difference, and embraces our entanglement in the material and social worlds we inhabit.

Notes and References

1
Brandon LaBelle, Background Noise: Perspectives on Sound Art, Bloomsbury Publishing USA, 2015, xi

2
This idea is drawn from the work of feminist science scholar Karen Barad, for whom discursive practices and material phenomena do not stand in a relationship of externality to each other; rather… matter and meaning are mutually articulated. Therefore, data or knowledge about particular phenomena is co-produced by the research apparatus, which we might understand as a “material-discursive” assemblage.
Karen Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Duke University Press, Durham & London, 2007, 146–153

3
Donna Haraway, “The Promises of Monsters: A Regenerative Politics for Inappropriate/d Others,” in Cultural Studies, Routledge, 1991, 295–337

4
Thematic analysis (TA) is a flexible method for coding qualitative data and organising those codes to identify themes: patterns of shared meaning that cut across interviews and capture what is significant in relation to the research aims.
Virginia Braun and Victoria Clarke, “Using Thematic Analysis in Psychology,” Qualitative Research in Psychology 3(2), 2006, 77–101

5
Micro-phenomenology is an interview method for eliciting fine-grained accounts of lived experience. Developed by Claire Petitmengin, it offers a way of capturing and analysing subjective experience, drawing on both phenomenology and neuroscience.
Claire Petitmengin, “Describing One’s Subjective Experience in the Second Person,” Phenomenology and the Cognitive Sciences 5(3–4), 2006, 229–269

6
Developed by Karen Barad, agential realism proposes a radically relational ontology in which bounded entities do not pre-exist their relations. Instead, phenomena emerge through intra-actions across human and material configurations.
Karen Barad, Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning, Duke University Press, Durham & London, 2007

7
Interaction analysis investigates how people make sense of one another’s actions and collaborate in situated contexts, emphasising the social and environmental dimensions of experience.
Brigitte Jordan and Austin Henderson, “Interaction Analysis: Foundations and Practice,” The Journal of the Learning Sciences 4(1), 1995, 39–103

8
Jackson and Mazzei draw on Deleuze and Guattari’s notion of “plugging in” as a way of thinking beyond conventional coding practices. “Plugging one text into another” produces diffractive assemblages, revealing data and theory as supple and capable of transforming one another (p. 267).
Alecia Y. Jackson and Lisa A. Mazzei, “Plugging One Text into Another,” Qualitative Inquiry 19(4), 2013, 261–271

9
For Salomé Voegelin, listening is a “knowing of the moment… that involves the listener and the sound in a reciprocal inventive production” (p. 5). She draws on Merleau-Ponty’s image of “being-honeyed,” where the act of grasping transforms both object and subject.
Salomé Voegelin, Listening to Noise and Silence: Towards a Philosophy of Sound Art, Bloomsbury Publishing USA, 2010, 9

10
Building on relational ontologies and new materialism, Christoph Cox proposes “sonic materialism” as a framework grounded in the immersive and vibrational nature of sound. Rather than fixed objects, he emphasises events, flux, and processes of becoming.
Christoph Cox, “Beyond Representation and Signification: Toward a Sonic Materialism,” Journal of Visual Culture 10(2), 2011, 145–161

11
Peter Szendy, Of Stigmatology: Punctuation as Experience, Fordham University Press, 2020

12
Salomé Voegelin, “Sonic Sense: The Meaning of the Invisible,” in The Oxford Handbook of Sound Art, Oxford University Press, 2021, 349–366

13
The design journey is an RtD method for documenting process and reflecting on the “direction and decisions taken to elicit insights,” bringing dispersed details into analytical relation (p. 793).
Ron Wakkary et al., “Turner Boxes and Bees: From Ambivalence to Diffraction,” in Proceedings of the 2023 ACM Designing Interactive Systems Conference, ACM, New York, 2023, 790–807

14
Attempting to eliminate unwanted self-noise, I sourced a different LiPo charger (Seeed Lipo Rider Plus2), which resolved the issue. With a cleaner output signal, I adjusted resistor and capacitor values in the amplification circuit to increase gain, enabling a louder and more transparent mediation of ultrasonic phenomena in the installation space.

Acknowledgements

The project was completed under the supervision of Professor Andrew McPherson and Professor Nick Bryan-Kinns. It was examined by Dr Sarah Fdili-Alaoui and Dr Anna Xambo in 2025.

The work is supported by the EPSRC grant EP/L01632X/1 (Centre for Doctoral Training in Media and Arts Technology at Queen Mary University of London) and by the Royal Academy of Engineering under the Research Chairs and Senior Research Fellowships scheme.