end of summer & dis 2024 ⛵️

Just a short summer recap: had a relaxing time and presented research on designing with electromyography (EMG) during some excellent time spent at the ACM International Conference on Designing Interactive Systems (DIS) - my first time at DIS! - in always lovely Copenhagen, DK, the capital of design~ ☀️

With the amazing Laia Turmo Vidal (left) and Atau Tanaka, even if only in our presentation (right)!

data entanglements

Andrew and I presented our paper, a collaboration with Landon Morrison, David Fierro, and Atau Tanaka during the conference:

Sonic Entanglements with Electromyography: Between Bodies, Signals, and Representations 🎻

In this paper we examine our expectations of data and how we disrupt the very act of making music through our attempts to measure it. We focus on EMG sensing in digital musical instrument design, a commonality between our work, showing how EMG sensing reconfigures musicians’ practice through its use, creating new interaction paradigms.

Presenting at the IT University of Copenhagen - - - in is the only way out.

new pubs

Two long-term projects I’ve collaborated on were finally published in August - it feels so good to have these available after so much work! PDFs for all of these papers are available below:

Book Chapter: Explainable AI and Music 👩🏼‍💻

With Nick Bryan-Kinns, Berker Banar, Corey Ford, Yixiao Zhang, and Jack Armitage:
Our chapter in the new volume Artificial Intelligence for Art Creation and Understanding by CRC Press examines AI practices in music and how we can design through the lens of performers and composers for understandability in using generative music tools.

Shifting Ambiguity, Collapsing Indeterminacy: Designing with Data as Baradian Apparatus 🎸

With Adan L. Benito, Franco Caspe, and Andrew McPherson:
Just accepted and soon-to-be published in the ACM Transactions on Computer Human Interaction (TOCHI) special issue on Prototyping with Uncertainty, we use Karen Barad’s agential reality and theory on meaning-making to demonstrate how we distill the messiness of our world into digital mappings and create new phenomena through the apparatus.

references

2024

Shifting Ambiguity, Collapsing Indeterminacy: Designing with Data as Baradian Apparatus

Courtney N. Reed, Adan L. Benito, Franco Caspe, and 1 more author

ACM Trans. Comput.-Hum. Interact., Aug 2024

Abs Bib PDF

This paper examines how digital systems designers distil the messiness and ambiguity of the world into concrete data that can be processed by computing systems. Using Karen Barad’s agential realism as a guide, we explore how data is fundamentally entangled with the tools and theories of its measurement. We examine data-enabled artefacts acting as Baradian apparatuses: they do not exist independently of the phenomenon they seek to measure, but rather collect and co-produce observations from within their entangled state: the phenomenon and the apparatus co-constitute one another. Connecting Barad’s quantum view of indeterminacy to the prevailing HCI discourse on the opportunities and challenges of ambiguity, we suggest that the very act of trying to stabilise a conceptual interpretation of data within an artefact has the paradoxical effect of amplifying and shifting ambiguity in interaction. We illustrate these ideas through three case studies from our own practices of designing digital musical instruments (DMIs). DMIs necessarily encode symbolic and music-theoretical knowledge as part of their internal operation, even as conceptual knowledge is not their intended outcome. In each case, we explore the nature of the apparatus, what phenomena it co-produces, and where the ambiguity lies to suggest approaches for design using these abstract theoretical frameworks.
@article{Reed_TOCHI_ShiftingAmbiguity, title = {{Shifting Ambiguity, Collapsing Indeterminacy: Designing with Data as Baradian Apparatus}}, author = {Reed, Courtney N. and Benito, Adan L. and Caspe, Franco and McPherson, Andrew P.}, year = {2024}, month = aug, journal = {ACM Trans. Comput.-Hum. Interact.}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, doi = {10.1145/3689043}, issn = {1073-0516}, url = {https://doi.org/10.1145/3689043}, }
Explainable AI and Music

Nick Bryan-Kinns, Berker Banar, Corey Ford, and 3 more authors

In Artificial Intelligence for Art Creation and Understanding, Jul 2024

Abs Bib PDF

The field of eXplainable Artificial Intelligence (XAI) has become a hot topic examining how machine learning models such as neural nets and deep learning techniques can be made more understandable to humans. However, there is very little research on XAI for the arts. This chapter explores what XAI might mean for AI and art creation by exploring the potential of XAI for music generation. One hundred AI and music papers are reviewed to illustrate how AI models are being explained, or more often not explained, and to suggest some ways in which we might design XAI systems to better help humans to get an understanding of what an AI model is doing when it generates music. Then the chapter demonstrates how a latent space model for music generation can be made more explainable by extending the MeasureVAE architecture to include explainable attributes in combination with offering real-time music generation. The chapter concludes with four key challenges for XAI for music and the arts more generally: i) the nature of explanation; ii) the effect of AI models, features, and training sets on explanation; iii) user centred design of XAI; iv) Interaction Design of explainable interfaces.
@incollection{BryanKinns_CRCPress_XAI, title = {{Explainable AI and Music}}, author = {Bryan-Kinns, Nick and Banar, Berker and Ford, Corey and Reed, Courtney N. and Zhang, Yixiao and Armitage, Jack}, year = {2024}, month = jul, booktitle = {{Artificial Intelligence for Art Creation and Understanding}}, publisher = {CRC Press}, pages = {1–29}, doi = {10.1201/9781003406273-1}, isbn = {9781003406273}, url = {http://dx.doi.org/10.1201/9781003406273-1}, }
Sonic Entanglements with Electromyography: Between Bodies, Signals, and Representations

Courtney N. Reed, Landon Morrison, Andrew P. McPherson, and 2 more authors

In Proceedings of the 2024 ACM Designing Interactive Systems Conference, Jul 2024

Abs Bib PDF

This paper investigates sound and music interactions arising from the use of electromyography (EMG) to instrumentalise signals from muscle exertion of the human body. We situate EMG within a family of embodied interaction modalities, where it occupies a middle ground, considered as a “signal from the inside” compared with external observations of the body (e.g., motion capture), but also seen as more volitional than neurological states recorded by brain electroencephalogram (EEG). To understand the messiness of gestural interaction afforded by EMG, we revisit the phenomenological turn in HCI, reading Paul Dourish’s work on the transparency of “ready-to-hand” technologies against the grain of recent posthumanist theories, which offer a performative interpretation of musical entanglements between bodies, signals, and representations. We take music performance as a use case, reporting on the opportunities and constraints posed by EMG in workshop-based studies of vocal, instrumental, and electronic practices. We observe that across our diverse range of musical subjects, they consistently challenged notions of EMG as a transparent tool that directly registered the state of the body, reporting instead that it took on “present-at-hand” qualities, defamiliarising the performer’s own sense of themselves and reconfiguring their embodied practice.
@inproceedings{Reed_DIS24_SonicEntanglements, title = {{Sonic Entanglements with Electromyography: Between Bodies, Signals, and Representations}}, author = {Reed, Courtney N. and Morrison, Landon and McPherson, Andrew P. and Fierro, David and Tanaka, Atau}, year = {2024}, month = jul, booktitle = {{Proceedings of the 2024 ACM Designing Interactive Systems Conference}}, location = {Copenhagen, Denmark}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, series = {DIS '24}, pages = {2691–2707}, numpages = {17}, doi = {10.1145/3643834.3661572}, isbn = {9798400705830}, url = {https://doi.org/10.1145/3643834.3661572}, }