Digital life stories: Semi-automatic (auto)biographies within lifelog collections

Our life stories enable us to reflect upon and share our personal histories. Through emerging digital technologies the possibility of collecting life experiences digitally is increasingly feasible; consequently so is the potential to create a digital counterpart to our personal narratives. In this work, lifelogging tools are used to collect digital artifacts continuously and passively throughout our day. These include images, documents, emails and webpages accessed; texts messages and mobile activity. This range of data when brought together is known as a lifelog. Given the complexity, volume and multimodal nature of such collections, it is clear that there are significant challenges to be addressed in order to achieve coherent and meaningful digital narratives of our events from our life histories. This work investigates the construction of personal digital narratives from lifelog collections. It examines the underlying questions, issues and challenges relating to construction of personal digital narratives from lifelogs. Fundamentally, it addresses how to organize and transform data sampled from an individual’s day-to-day activities into a coherent narrative account. This enquiry is enabled by three 20-month long-term lifelogs collected by participants and produces a narrative system which enables the semi-automatic construction of digital stories from lifelog content. Inspired by probative studies conducted into current practices of curation, from which a set of fundamental requirements are established, this solution employs a 2-dimensional spatial framework for storytelling. It delivers integrated support for the structuring of lifelog content and its distillation into storyform through information retrieval approaches. We describe and contribute flexible algorithmic approaches to achieve both. Finally, this research inquiry yields qualitative and quantitative insights into such digital narratives and their generation, composition and construction. The opportunities for such personal narrative accounts to enable recollection, reminiscence and reflection with the collection owners are established and its benefit in sharing past personal experience experiences is outlined. Finally, in a novel investigation with motivated third parties we demonstrate the opportunities such narrative accounts may have beyond the scope of the collection owner in: personal, societal and cultural explorations, artistic endeavours and as a generational heirloom

Communicating with your E-memory: finding and refinding in personal lifelogs

The rapid development of technology enables the digital capture and storage of our life experiences in an “E-Memory” (electronic–memory) or personal lifelog (PLL). This offers the potential for people to store the details of their life in a permanent archive, so that the information is still available even when its physical existence has vanished and when memory traces of it have faded away. A major challenge for PLLs is enabling people to access information when it is needed. Many people may also want to share or transfer some of their memory to their friends and descendants, so that their experiences can be appreciated and their knowledge can be kept even after they have passed away. This thesis further explores people’s potential needs from their own PLLs, discuss the possible methods people may use and potential problems that they may encounter while accessing their PLLs, and hypothesize that better support of users’ own memory can provide better user experience and improved efficiency for accessing their E-memories (or PLLs). As part of a larger project, three lifeloggers collected their own prototype lifelog collection for about 20 months’ time. To complete this study, the author developed a prototype PLL system, called the iCLIPS Lifelog Archive Browser (LAB), based on the author’s theoretical exploration and empirical studies, and evaluated it using our prototype lifelog collections through a user study with the three lifeloggers. The results of this study provide promising evidence which support the hypothesis. The end of this thesis also discusses the issues that the lifeloggers encountered in using their lifelogs and future technologies that are desirable based the studies in this thesis

Providing effective memory retrieval cues through automatic structuring and augmentation of a lifelog of images

Lifelogging is an area of research which is concerned with the capture of many aspects of an individual's life digitally, and within this rapidly emerging field is the significant challenge of managing images passively captured by an individual of their daily life. Possible applications vary from helping those with neurodegenerative conditions recall events from memory, to the maintenance and augmentation of extensive image collections of a tourist's trips. However, a large lifelog of images can quickly amass, with an average of 700,000 images captured each year, using a device such as the SenseCam. We address the problem of managing this vast collection of personal images by investigating automatic techniques that: 1. Identify distinct events within a full day of lifelog images (which typically consists of 2,000 images) e.g. breakfast, working on PC, meeting, etc. 2. Find similar events to a given event in a person's lifelog e.g. "show me other events where I was in the park" 3. Determine those events that are more important or unusual to the user and also select a relevant keyframe image for visual display of an event e.g. a "meeting" is more interesting to review than "working on PC" 4. Augment the images from a wearable camera with higher quality images from external "Web 2.0" sources e.g. find me pictures taken by others of the U2 concert in Croke Park In this dissertation we discuss novel techniques to realise each of these facets and how effective they are. The significance of this work is not only of benefit to the lifelogging community, but also to cognitive psychology researchers studying the potential benefits of lifelogging devices to those with neurodegenerative diseases

The Songs of Our Past

Advancements in technology have resulted in unique changes in the way people interact with music today: Small, portable devices allow listening to it everywhere and provide access to thousands or, via streaming, even millions of songs. In addition, all played tracks can be logged with an accuracy down to the second. So far, these music listening histories are mostly used for music recommendation and hidden from their actual creators. But people may also benefit from this data more directly: as memory extensions that allow retrieving the name of a title, for rediscovering old favorites and reflecting about their lives. Additionally, listening histories can be representations of the implicit relationships between musical items. In this thesis, I discuss the contents of these listening histories and present software tools that give their owners the chance to work with them. As a first approach to understanding the patterns contained in listening histories I give an overview of the relevant literature from musicology, human-computer-interaction and music information retrieval. This literature review identifies the context as a main influence for listening: from the musical and temporal to the demographical and social. I then discuss music listening histories as digital memory extensions and a part of lifelogging data. Based on this notion, I present what an ideal listening history would look like and how close the real-world implementations come. I also derive a design space, centered around time, items and listeners, for this specific type of data and shortcomings of the real-world data regarding the previously identified contextual factors. The main part of this dissertation describes the design, implementation and evaluation of visualizations for listening histories. The first set of visualizations presents listening histories in the context of lifelogging, to allow analysing one’s behavior and reminiscing. These casual information visualizations vary in complexity and purpose. The second set is more concerned with the musical context and the idea that listening histories also represent relationships between musical items. I present approaches for improving music recommendation through interaction and integrating listening histories in regular media players. The main contributions of this thesis to HCI and information visualization are: First, a deeper understanding of relevant aspects and important patterns that make a person’s listening special and unique. Second, visualization prototypes and a design space of listening history visualizations that show approaches how to work with temporal personal data in a lifelogging context. Third, ways to improve recommender systems and existing software through the notion of seeing relationships between musical items in listening histories. Finally, as a meta-contribution, the casual approach of all visualizations also helps in providing non-experts with access to their own data, a future challenge for researchers and practitioners alike

Blended memory: distributed remembering and forgetting through digital photography

This thesis explores practices and experiences of using photography to support remembering. While the increasing use of photography is well documented, we have limited theoretical understanding of how we approach the taking, organising, and sharing of personal images in relation to memory, and of the opportunities and risks that are created through technological change. Two studies were conducted in which a total of 21 participants were interviewed in front of a sample of their photographs. Study 1 explored photography and remembering around a single, specific event: a wedding. Study 2 explored longer-term patterns of photographic and remembering activity across a range of contexts and events. The analysis showed that the ways that participants engaged with other people and technologies were significant in determining the kinds of photographs that were produced, and the engagement with those photos. Photographic practices were also heavily influenced by the situations in which they were performed and the beliefs and preferences of individuals. The existence of photographs could lead to thinking about particular aspects of the past, but the taking of photographs also altered the experience of what was being photographed. This could be seen as disruptive, depending on the participant’s beliefs about whether photography was a legitimate part of experience. When taking photos, participants pursued a mix of aesthetics, objectivity, and personal meaning, and perceptions of these qualities could influence the way that photographs were used in cueing recall. However, while most participants had produced large collections of photographs, there had been limited engagement with these and taking or having photographs could be more important than looking at them. The thesis concludes that there is value in redefining memory as a kind of activity that emerges through the performance of remembering and that is dependent on the tools used to support it and the situations in which it is performed. From this perspective, photography and autobiographical remembering are parts of the same wider activity, an inseparable blend of internal and external processes. As such, attempts to support our memories should consider both the features of technology and the experience of using it, as well as the ways that we work with tools and people when remembering

Re-thinking lifelogging : designing human-centric prosthetic memory devices.

Building Prosthetic Memory (PM) technology has been an active research area for the past few decades, with the primary aim in supporting Organic Memory (OM) in remembering everyday events and experiences. Through building and evaluating new PM tools, this thesis attempts to explore how and when PM tools are used to help OM in everyday memory tasks. The focus of this thesis is to investigate PM tools as an extension of, or a supplement to, OM and to understand why people choose to use PM as opposed to their OM to help them retrieve information. Further aims of this thesis are to investigate the role of Metamemory and social processes. Finally, the work aims to support Autobiographical memory through building new PM tools. The studies apply mixed experimental and naturalistic methods, and include 3 controlled lab studies and 3 field trials involving a total of 217 participants. Overall, there were 5 new PM devices built and evaluated in long-term and controlled contexts. Results obtained through lab studies suggest that PM and OM function in a synergetic relationship. In particular, use of PM increases when OM is particularly weak and this interaction is mediated by organic Metamemory processes. PM properties also have an influence - people prefer efficient over accurate PM devices. Furthermore, PM cues help in two ways: 1) at encoding to help focus OM; and 2) at retrieval to cue partially remembered information. Longer term studies also reveal that PM is not used to substitute for OM. Instead users prefer to use recordings to access specific parts of a lecture rather than listen to the whole thing. Such tools are extensively used by non-native speakers, although only native speakers' coursework benefits from usage. PM tools that support social summarisation demonstrate that people exploit social feedback and cues provided by other users and that these improve recall. IV Finally, evaluations of new autobiographical memory tools show that people upload mementos based on their importance. There is evidence for preference for mementos that are associated with other people and home. I conclude with a discussion of the design and theory implications of this work

Designing and evaluating a user interface for continous embedded lifelogging based on physical context

PhD ThesisAn increase in both personal information and storage capacity has encouraged people to store and archive their life experience in multimedia formats. The usefulness of such large amounts of data will remain inadequate without the development of both retrieval techniques and interfaces that help people access and navigate their personal collections. The research described in this thesis investigates lifelogging technology from the perspective of the psychology of memory and human-computer interaction. The research described seeks to increase my understanding of what data can trigger memories and how I might use this insight to retrieve past life experiences in interfaces to lifelogging technology. The review of memory and previous research on lifelogging technology allows and support me to establish a clear understanding of how memory works and design novel and effective memory cues; whilst at the same time I critiqued existing lifelogging systems and approaches to retrieving memories of past actions and activities. In the initial experiments I evaluated the design and implementation of a prototype which exposed numerous problems both in the visualisation of data and usability. These findings informed the design of novel lifelogging prototype to facilitate retrieval. I assessed the second prototype and determined how an improved system supported access and retrieval of users’ past life experiences, in particular, how users group their data into events, how they interact with their data, and the classes of memories that it supported. In this doctoral thesis I found that visualizing the movements of users’ hands and bodies facilitated grouping activities into events when combined with the photos and other data captured at the same time. In addition, the movements of the user's hand and body and the movements of some objects can promote an activity recognition or support user detection and grouping of them into events. Furthermore, the ability to search for specific movements significantly reduced the amount of time that it took to retrieve data related to specific events. I revealed three major strategies that users followed to understand the combined data: skimming sequences, cross sensor jumping and continued scanning

Case studies in therapeutic SenseCam use aimed at identity maintenance in early stage dementia

In the absence of a medical cure for memory loss new technologies specialised in pervasive imaging are being incorporated into interventions for dementia. The practice of lifelogging is a digital capture of life experiences typically through mobile devices such as SenseCam. The lightweight wearable digital camera passively captures about 3,000 images a day. Lifelogging results in personal, recent prompts, potentially encouraging sharing of personal memories. This research investigated the incorporation of lifelogging technology into a therapeutic approach aimed to support people with dementia by using the Case Study method, an exploratory and descriptive approach. The case study is a method of empirical inquiry that enables investigation of phenomenon within its real life context. SenseCam therapy aimed to stimulate the cognition of a person with dementia, with support of their personal identity as its primary goal. SenseCam images were used as cues to meaningful discussions about the person’s recent memories. The images enabled a construction of a particular version of the participants’ identities mainly based in their recent past. On the contrary participants seemed to valorise their identity of their distant past. The SenseCam identity also contained uncensored details from participants’ lives as revealed by review of SenseCam images. The exposing nature of SenseCam images posed risks to the users’ privacy and showed the potential ethical risks of using lifelogging technology with people with dementia. There is limited literature on the practical recommendations on how to use lifelogging devices and how they affect people with dementia. The results from this research indicate that a number of factors should be considered when using lifelogging technology with people with dementia. Firstly the contextual factors of people with dementia including the level of cognitive impairment, existing coping mechanisms and the interaction patterns with the carer need to be considered. Secondly the technology should be used within a therapeutic framework and tailored to suit the individual needs of both people with dementia and their carers. Lastly intimate and unexpected details from the participant’s life should be discussed in an ethical and sensitive manner. Implications of not working within these boundaries show clear potential for undermining the human rights and potentially the wellbeing of people with dementia

Mobile technologies for chronic condition management.

The management of long term chronic conditions is a complex and challenging task. The process relies on individuals engaging in regular recording of factors that affect their health. Yet currently, the mobile tools that people carry with them are not being fully utilised to assist in this process. This Thesis reports on research that has been completed to understand the role that mobile technologies can have in supporting people with chronic conditions. An individual engaging in personal monitoring is concerned with the data they collect, not the process used to capture the data. The results of the research conducted contribute to an advancement of knowledge around how mobile technologies can assist in personal reflection on health information to provide greater understanding of chronic disease management This understanding of the role of reflection in chronic condition management can then be used as a platform to improve the mobile interventions in future implementations. These findings are arrived at by conducting an initial investigation into the usage of existing health monitoring devices and an evaluation of these devices is detailed. The results of this early work suggests there exists a gap between real practice and the role that mobile technologies can play in assisting with the process. A deeper understanding of the management practices of people with diabetes is then achieved through a set of interviews with individuals with diabetes. The findings then define a model of chronic disease management, named the 'Diabetes Management Cycle.' Following the definition of the cycle, a mobile application was implemented and deployed during a four week evaluation with individuals with type 1 diabetes. This system was designed to support existing management practices and implemented simple methods of information capture. A second application was then developed to enable increased monitoring and subsequent reflection amongst individuals with cardiovascular conditions. The application was deployed in a six week in-situ evaluation and it was discovered a personalised 'tagging' mechanism allowed for the discovery of patterns affecting health. Based on the findings of the studies, the Thesis concludes by presenting definitions of ready- to-hand in the short- and long-term contexts of mobile health management. These ready-to-hand guidelines provide a platform for future research projects to build upon