top of page

Sonic Proxemics with AI

Immersive Theatre

The Hidden Symphony: The Intimacy and Playful Politics in Mixed Reality

Research brief

Using COVID as a catalyst, this study proposes the immersive theatre approach, aiming to rebuild social closeness, raise awareness of AI applications, and rethink how we interact with others in public spaces from now on.

By integrating motion detection and biophysical sensors in smart suits as interactive triggers, this research explores new methods to enhance well-being. It focuses on the dynamic relationships between humans and non-organic entities such as robots and AI agents in mixed reality environments, allowing participants to engage through 'tele-presence'. By uncovering the invisible social connections—how individuals associate or isolate from each other—this study aims to contribute to public mental health, offering a playful and innovative approach to bringing people together.

The Abstract, Introduction and Objectives of this study are available for download here (EN+CN).

For full reference of the current study, please scroll down to the Bibliography & Credit section.

Images from the corona era(Photo_Reuters China Daily CDIC).jpeg
OECD2022_brown.PNG

Rising Trends in Depression 

Recent developments in mental health care have heightened the need for re-examining the social support system and the influence of isolation. The 2022 OECD Health Statistics indicate a twofold increase in symptoms of depression in several European countries during the pandemic, attributed in part to anti-COVID policies like mandatory social distancing and quarantine, leading to heightened social isolation, a known precursor to depression (see Figure 1) [1].

 

Similarly, Balakrishnan et al.’s investigation on COVID-19 depression in Asia-Pacific shows that the isolation-related risk factor (i.e., ‘living alone/isolation’ and ‘lack of social support/community support’) is the third highest, following ‘fear of COVID-19 infection’ and ‘gender’ [19]. According to OECD report, social isolation is one of the main factors resulting in the increased spread of depression and anxiety across all age groups [1].

Proximity Culture

Embracing Tele-Presence : An Underrated Layer in Public Space.

As societal dynamics shift, online meetings have become the predominant mode of communication, altering the notions of 'being there' and 'being with someone.' In response, this study seeks to reevaluate Edward T. Hall’s proxemics model within the context of post-COVID life, where digital interactions hold equal importance to in-person relationships.

 

Several studies investigating proxemics have been carried out on media and communication fields. However, far too little attention has been paid to ‘tele-presence’ or ‘tele-participation’, which shakes the definition of ‘space’ in the traditional sense. As highlighted by Drąg (2020), failure to consider this transformation of the role of space in communication using modern technologies can increasingly weaken the cognitive value of traditional communication models [2].

Image1: Following the advice from the South Korean government, public and private institutions introduced exams that adhere to social distancing. 

06-covid-south-korea-exam.jpg
gatebox1.png

Image 2: Hikari Azuma, powered by AI hologram technologies developed by Gatebox Inc., functions as a virtual home assistant dedicated to offering a comforting presence for individuals living alone.

Non-human Companions

The increasing presence of social robots, AI agents, autonomous machines. 

Moreover, the integration of social robots and artificial intelligent companions into daily life, as envisioned by Bruno Latour and Donna Haraway, highlights the growing presence of non-human entities. This is supported by Novice and Rodriguez’s study (2021) which reveals that “while the real-world proxemics of human-human interaction have been well studied, the virtual-world proxemics of human-agent interaction are less well understood” [13]. Current literature includes only a few studies on 'proxemic interaction' and its visualisation, with a noticeable dearth of research on audio-related aspects.

 

The study by Ghaimi et al. (2022) examined the application of proxemic map in virtual reality which points toward that “using proxemics to perform new interactions in an immersive environment has not been systematically explored" [14]. In addition, study by Camara and Fox (2022) found that “robot navigation in environments with static objects is largely solved, but navigating around humans in dynamic environments remains an active research question for autonomous vehicles (AVs)” [9].

Dynamic Proxemics

Playful Politics Between Non-humans and Humans.

In other words, Hall's proxemics model has become insufficient for explaining the evolving interaction dynamics between humans and objects. “The proxemic utility needs to be given in the form of continuous functions corresponding to the continuous motion of people. A review of the literature showed that there is no available method to infer the continuous proxemic functions,” indicated Camara and Fox. Thus, this study incorporates autonomous robots and Artificial Intelligence (AI) agents into the experimental design, framing it as immersive theatre with a specific emphasis on sound. This approach aims to develop a more comprehensive understanding of proxemics, highlighting the continuous interactions among individuals in phygital spaces that mirror our mixed reality.

Immersive Theatre_Sean Pollock Photography1.jpg

Image 3: Immersive Theatre〈Séance〉by Darkfield, photo by Sean Pollock.

Supervisors: Dr. Sam Redfern (School of Computer Science) & Dr. Máiréad Ní Chróinín (School of English and Creative Arts). Full academic reference of current study can be accessed here.

Bibliography

(selected)

[1] OECD/European Union (2022), Health at a Glance: Europe 2022: State of Health in the EU Cycle, OECD Publishing, Paris, DOI (p59).

[2] Drąg, K. (2020). Revaluation of The Proxemics Code in Mediated Communication. Social Communication, vol.1., pp93-105, DOI (p103).

[9] Camara F., Fox C. (2022) Unfreezing autonomous vehicles with game theory, proxemics, and trust.
 Front. Comput. Sci. 4:969194., DOI.

 

[13] Novick, D., Rodriguez, A.E. (2021). A Comparative Study of Conversational Proxemics for Virtual Agents. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality. HCII 2021. Lecture Notes in Computer Science(), vol 12770. Springer, Cham. DOI.

 

[14] Ghaemi,Z., Engelke, U., Ens, B. & Jenny, B. (2022) Proxemic maps for immersive visualization, Cartography and Geographic Information Science, 49:3, 205-219, DOI.

[19] Balakrishnan, V., Ng, K. S., Kaur, W., Govaichelvan, K., & Lee, Z. L. (2022). Covid-19 depression and its risk factors in Asia Pacific – a systematic review and meta-analysis. Journal of Affective Disorders, 298, 47–56, DOI.

Credit

Image 1. Zastrow, M., & Park, J. M. (2020, May 12). How South Korea prevented a coronavirus disaster-and why the battle isn’t over. National Geographic. Link

Image 2. キャラクター召喚装置「Gatebox」. Gatebox株式会社. (n.d.). Link.

Image 3. Clarke, M. (2020, March 2). Darkfield: Immersive theatre brings millennial ghost train vibe to king’s cross. Londonist. Link

Soundtrack: Angels By My Side, by Lunar Years

Music: bensound.com

License code: RHWFATTAK9CPY2YR

Acoustic Intimacy.gif
bottom of page