Designing and evaluating a user interface for continous embedded lifelogging based on physical context

PhD ThesisAn increase in both personal information and storage capacity has encouraged people to store and archive their life experience in multimedia formats. The usefulness of such large amounts of data will remain inadequate without the development of both retrieval techniques and interfaces that help people access and navigate their personal collections. The research described in this thesis investigates lifelogging technology from the perspective of the psychology of memory and human-computer interaction. The research described seeks to increase my understanding of what data can trigger memories and how I might use this insight to retrieve past life experiences in interfaces to lifelogging technology. The review of memory and previous research on lifelogging technology allows and support me to establish a clear understanding of how memory works and design novel and effective memory cues; whilst at the same time I critiqued existing lifelogging systems and approaches to retrieving memories of past actions and activities. In the initial experiments I evaluated the design and implementation of a prototype which exposed numerous problems both in the visualisation of data and usability. These findings informed the design of novel lifelogging prototype to facilitate retrieval. I assessed the second prototype and determined how an improved system supported access and retrieval of users’ past life experiences, in particular, how users group their data into events, how they interact with their data, and the classes of memories that it supported. In this doctoral thesis I found that visualizing the movements of users’ hands and bodies facilitated grouping activities into events when combined with the photos and other data captured at the same time. In addition, the movements of the user's hand and body and the movements of some objects can promote an activity recognition or support user detection and grouping of them into events. Furthermore, the ability to search for specific movements significantly reduced the amount of time that it took to retrieve data related to specific events. I revealed three major strategies that users followed to understand the combined data: skimming sequences, cross sensor jumping and continued scanning

Communicating with your E-memory: finding and refinding in personal lifelogs

The rapid development of technology enables the digital capture and storage of our life experiences in an “E-Memory” (electronic–memory) or personal lifelog (PLL). This offers the potential for people to store the details of their life in a permanent archive, so that the information is still available even when its physical existence has vanished and when memory traces of it have faded away. A major challenge for PLLs is enabling people to access information when it is needed. Many people may also want to share or transfer some of their memory to their friends and descendants, so that their experiences can be appreciated and their knowledge can be kept even after they have passed away. This thesis further explores people’s potential needs from their own PLLs, discuss the possible methods people may use and potential problems that they may encounter while accessing their PLLs, and hypothesize that better support of users’ own memory can provide better user experience and improved efficiency for accessing their E-memories (or PLLs). As part of a larger project, three lifeloggers collected their own prototype lifelog collection for about 20 months’ time. To complete this study, the author developed a prototype PLL system, called the iCLIPS Lifelog Archive Browser (LAB), based on the author’s theoretical exploration and empirical studies, and evaluated it using our prototype lifelog collections through a user study with the three lifeloggers. The results of this study provide promising evidence which support the hypothesis. The end of this thesis also discusses the issues that the lifeloggers encountered in using their lifelogs and future technologies that are desirable based the studies in this thesis

Re-thinking lifelogging : designing human-centric prosthetic memory devices.

Building Prosthetic Memory (PM) technology has been an active research area for the past few decades, with the primary aim in supporting Organic Memory (OM) in remembering everyday events and experiences. Through building and evaluating new PM tools, this thesis attempts to explore how and when PM tools are used to help OM in everyday memory tasks. The focus of this thesis is to investigate PM tools as an extension of, or a supplement to, OM and to understand why people choose to use PM as opposed to their OM to help them retrieve information. Further aims of this thesis are to investigate the role of Metamemory and social processes. Finally, the work aims to support Autobiographical memory through building new PM tools. The studies apply mixed experimental and naturalistic methods, and include 3 controlled lab studies and 3 field trials involving a total of 217 participants. Overall, there were 5 new PM devices built and evaluated in long-term and controlled contexts. Results obtained through lab studies suggest that PM and OM function in a synergetic relationship. In particular, use of PM increases when OM is particularly weak and this interaction is mediated by organic Metamemory processes. PM properties also have an influence - people prefer efficient over accurate PM devices. Furthermore, PM cues help in two ways: 1) at encoding to help focus OM; and 2) at retrieval to cue partially remembered information. Longer term studies also reveal that PM is not used to substitute for OM. Instead users prefer to use recordings to access specific parts of a lecture rather than listen to the whole thing. Such tools are extensively used by non-native speakers, although only native speakers' coursework benefits from usage. PM tools that support social summarisation demonstrate that people exploit social feedback and cues provided by other users and that these improve recall. IV Finally, evaluations of new autobiographical memory tools show that people upload mementos based on their importance. There is evidence for preference for mementos that are associated with other people and home. I conclude with a discussion of the design and theory implications of this work

Digital life stories: Semi-automatic (auto)biographies within lifelog collections

Our life stories enable us to reflect upon and share our personal histories. Through emerging digital technologies the possibility of collecting life experiences digitally is increasingly feasible; consequently so is the potential to create a digital counterpart to our personal narratives. In this work, lifelogging tools are used to collect digital artifacts continuously and passively throughout our day. These include images, documents, emails and webpages accessed; texts messages and mobile activity. This range of data when brought together is known as a lifelog. Given the complexity, volume and multimodal nature of such collections, it is clear that there are significant challenges to be addressed in order to achieve coherent and meaningful digital narratives of our events from our life histories. This work investigates the construction of personal digital narratives from lifelog collections. It examines the underlying questions, issues and challenges relating to construction of personal digital narratives from lifelogs. Fundamentally, it addresses how to organize and transform data sampled from an individual’s day-to-day activities into a coherent narrative account. This enquiry is enabled by three 20-month long-term lifelogs collected by participants and produces a narrative system which enables the semi-automatic construction of digital stories from lifelog content. Inspired by probative studies conducted into current practices of curation, from which a set of fundamental requirements are established, this solution employs a 2-dimensional spatial framework for storytelling. It delivers integrated support for the structuring of lifelog content and its distillation into storyform through information retrieval approaches. We describe and contribute flexible algorithmic approaches to achieve both. Finally, this research inquiry yields qualitative and quantitative insights into such digital narratives and their generation, composition and construction. The opportunities for such personal narrative accounts to enable recollection, reminiscence and reflection with the collection owners are established and its benefit in sharing past personal experience experiences is outlined. Finally, in a novel investigation with motivated third parties we demonstrate the opportunities such narrative accounts may have beyond the scope of the collection owner in: personal, societal and cultural explorations, artistic endeavours and as a generational heirloom

Memory as Concept and Design in Digital Recording Devices

This thesis focuses on scientists and technologies brought together around the desire to improve fallible human memory. Based on extended ethnographic fieldwork, it considers interdisciplinary collaborations among experts who design recording and archiving technologies that seek to maintain, extend, and commemorate life. How are everyday experiences translated as information, and for what purpose? How are our habits of drinking tea, talking on the phone, driving to work, and reminiscing with old photographs, turned into something that can be stored, analyzed and acted upon? How might information be used in real time to supplement the living in a recursive feedback loop? By addressing these questions, I reveal how these memory banks are inherently tied to logics of capital, of stock and storage, and to logics of the technological where, when it comes to memory, more is more. The first sections that make up this dissertation shift in scale from the micro to the macro: from historical national endeavors that turned ordinary citizens into a sensors and collectors of the mundane, to contemporary computational projects designed to store, organize and retrieve vast amounts of information. The second half of this dissertation focuses on two extreme cases of lifelogging that make use of prototypical recording technologies: Gordon Bell, who is on a quest to record his life for the sake of increased objectivity, productivity, and digital posterity, and Mrs. B, a woman who suffers from amnesia and records her life in the hopes of leading a normal life in which she can share the past with loved ones. Through these case studies, I show how new recording technologies are both a symptom of, and a cure for, anxieties about time. By focusing on the design of new objects and by addressing contemporary debates on the intentions that govern the making of recording machines, I examine how technologies take shape, and how they inform understandings of memory and the self as well as notions of human disability and enhancement. In short, I show that the past, as well as the present and the future, are always discursively, practically, and technologically informed