AUGSOC ERC Starter Grant Project

Everyday Augmented Reality (AR) headsets come with an array of sensing capabilities. Users wearing these headsets for extended periods may prefer specific sensors to remain inactive in some contexts for privacy and sensitivity reasons. Currently, the contexts in which users wish to limit sensor data collection are unclear. To explore this, we conducted a survey (\mathrmN=100), collecting 552 scenarios to understand which situations users wish to restrict or completely block data collection by specific sensors or combinations on their AR headset. Our results show the sensitive contexts can be classified into seven categories: 1) presence of confidential information; 2) risk of data quantification; 3) expectation of solitude; 4) rules prohibiting data collection; 5) modesty and nudity; 6) home environments; and 7) outdoor public locations. Our results provide insights into privacy-invasive contexts when people want to limit and restrict their AR sensors, building towards automating permission configurations during the prolonged use of everyday AR headsets.

With the advent of everyday Augmented Reality (AR) glasses the public will be able to receive personalised and context-relevant spatial information displays driven by complex Artificial Intelligence (AI) processes. Research on other everyday technologies shows that use can lead to beneficial or detrimental effects on human memory, depending on the use case or individual. Therefore, it is imperative to understand how everyday AR+AI might be used for memory purposes ahead of time, to ensure the creation of responsible memory augmentations: maximising benefits whilst minimising harms. Through a survey (N=120) we captured how likely respondents would be to use AR+AI devices for a list of 25 memory purposes derived from the literature and commercial materials. Our results predict frequent, pervasive and varied use of AR+AI across memory types, and that individual differences predict specific constellations or patterns of use. We provide recommendations for research areas that academia, industry and policymakers should prioritise.

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

Everyday Augmented Reality (AR) displays, with wearable, fashionable, all-day form factors, may one day supplant our reliance on physical displays, heralding new capabilities in augmented intelligence and perception, communication, productivity, and more. Such technology has the potential to become as fundamental to our daily lives as smartphones are today, empowering users, communities, business, governments, and others to alter, augment, diminish or otherwise mediate our perception of reality. For social good, this technology can enable augmenting expression of social identity to better represent our ‘authentic’ self, and virtually enhancing real-world social spaces to encourage greater community ownership and social cohesion. For social harm however, everyday AR could facilitate and amplify manipulation, information disorder (e.g. dis-information), censorship and coercion in our day-to-day experience of reality. In this essay, we consider some of the key societal changes (and ethical challenges) posed by the adoption of everyday AR, and argue that everyday AR will provoke the need for new human rights to be considered alongside proposed neurorights and existing and envisaged digital human rights, around: who can mediate reality (perceptual autonomy); what elements of reality are permissible to alter/augment (perceptual agency); and governing permissible intent regarding why we augment the user’s perception of reality, in particular considering tensions in cognitive autonomy (e.g. manipulation) and perceptual integrity (e.g. information disorder).