AUGSOC ERC Starter Grant Project

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

Mass adoption of Everyday Augmented Reality (AR) glasses will enable pervasive augmentation of our expression of social identity through AR filters, transforming our perception of self and others. However, despite filters’ prominent and often problematic usage in social media, research has yet to reflect on the potential impact AR filters might have when brought into everyday life. Informed by our survey of 300 existing popular AR filters used on Snapchat, Instagram and Tiktok, we conducted an AR-in-VR user study where participants (N=24) were exposed to 18 filters across six categories. We evaluated the social acceptability of these augmentations around others and attitudes towards an individual’s augmented self.Our findings highlight 1) how users broadly respected another individual’s augmented self; 2) positive use cases, such as supporting the presentation of gender identity; and 3) tensions around applying AR filters to others (e.g. censorship, changing protected characteristics) and their impact on self-perception (e.g. perpetuating unrealistic beauty standards). We raise questions regarding the rights of individuals to augment and be augmented that provoke the need for further consideration of AR augmentations in society.

Everyday Augmented Reality (AR) displays, with wearable, fashionable, all-day form factors, may one day supplant our reliance on physical displays, heralding new capabilities in augmented intelligence and perception, communication, productivity, and more. Such technology has the potential to become as fundamental to our daily lives as smartphones are today, empowering users, communities, business, governments, and others to alter, augment, diminish or otherwise mediate our perception of reality. For social good, this technology can enable augmenting expression of social identity to better represent our ‘authentic’ self, and virtually enhancing real-world social spaces to encourage greater community ownership and social cohesion. For social harm however, everyday AR could facilitate and amplify manipulation, information disorder (e.g. dis-information), censorship and coercion in our day-to-day experience of reality. In this essay, we consider some of the key societal changes (and ethical challenges) posed by the adoption of everyday AR, and argue that everyday AR will provoke the need for new human rights to be considered alongside proposed neurorights and existing and envisaged digital human rights, around: who can mediate reality (perceptual autonomy); what elements of reality are permissible to alter/augment (perceptual agency); and governing permissible intent regarding why we augment the user’s perception of reality, in particular considering tensions in cognitive autonomy (e.g. manipulation) and perceptual integrity (e.g. information disorder).