Designing and evaluating a user interface for continous embedded lifelogging based on physical context

PhD ThesisAn increase in both personal information and storage capacity has encouraged people to store and archive their life experience in multimedia formats. The usefulness of such large amounts of data will remain inadequate without the development of both retrieval techniques and interfaces that help people access and navigate their personal collections. The research described in this thesis investigates lifelogging technology from the perspective of the psychology of memory and human-computer interaction. The research described seeks to increase my understanding of what data can trigger memories and how I might use this insight to retrieve past life experiences in interfaces to lifelogging technology. The review of memory and previous research on lifelogging technology allows and support me to establish a clear understanding of how memory works and design novel and effective memory cues; whilst at the same time I critiqued existing lifelogging systems and approaches to retrieving memories of past actions and activities. In the initial experiments I evaluated the design and implementation of a prototype which exposed numerous problems both in the visualisation of data and usability. These findings informed the design of novel lifelogging prototype to facilitate retrieval. I assessed the second prototype and determined how an improved system supported access and retrieval of users’ past life experiences, in particular, how users group their data into events, how they interact with their data, and the classes of memories that it supported. In this doctoral thesis I found that visualizing the movements of users’ hands and bodies facilitated grouping activities into events when combined with the photos and other data captured at the same time. In addition, the movements of the user's hand and body and the movements of some objects can promote an activity recognition or support user detection and grouping of them into events. Furthermore, the ability to search for specific movements significantly reduced the amount of time that it took to retrieve data related to specific events. I revealed three major strategies that users followed to understand the combined data: skimming sequences, cross sensor jumping and continued scanning

Geographical places as a personalisation element: extracting profiles from human activities and services of visited places in mobility logs

Collecting personal mobility traces of individuals is currently applicable on a large scale due to the popularity of position-aware mobile phones. Statistical analysis of GPS data streams, collected with a mobile phone, can reveal several interesting measures such as the most frequently visited geographical places by some individual. Applying probabilistic models to such data sets can predict the next place to visit, and when. Several practical applications can utilise the results of such analysis. Current state of the art, however, is limited in terms of the qualitative analysis of personal mobility logs. Without explicit user-interactions, not much semantics can be inferred from a GPS log. This work proposes the utilisation of the common human activities and services provided at certain place types to extract semantically rich profiles from personal mobility logs. The resulting profiles include spatial, temporal and generic thematic description of a user. The work introduces several pre-processing methods for GPS data streams, collected with personal mobile devices, which improved the quality of the place extraction process from GPS logs. The thesis also introduces a method for extracting place semantics from multiple data sources. A textual corpus of functional descriptions of human activities and services associated with certain geographic place types is analysed to identify the frequent linguistic patterns used to describe such terms. The patterns found are then matched against multiple textual data sources of place semantics, to extract such terms, for a collection of place types. The results were evaluated in comparison to an equivalent expert ontology, as well as to semantics collected from the general public. Finally, the work proposes a model for the resulting profiles, the necessary algorithms to build and utilise such profiles, along with an encoding mark-up language. A simulated mobile application was developed to show the usability and for evaluation of the resulting profiles

Personalized Life Log Media System in Ubiquitous Environment

Abstract. In this paper, we propose new system for storing and retrieval of personal life log media on ubiquitous environment. We can gather personal life log media from intelligent gadgets which are connected with wireless network. Our intelligent gadgets consist of wearable gadgets and environment gadgets. Wearable gadgets include audiovisual device, GPS, 3D-accelerometer and physiological reaction sensors. Environment gadgets include the smart sensors attached to the daily supplies, such as cup, chair, door and so on. User can get multimedia stream with wearable intelligent gadget and also get the environmental information around him from environment gadgets as personal life log media. These life log media(LLM) can be logged on the LLM server in realtime. In LLM server, learning-based activity analysis engine will process logged data and create meta data for retrieval automatically. By using proposed system, user can log with personalized life log media and can retrieve the media at any time. To give more intuitive retrieval, we provide intuitive spatiotemporal graphical user interface in client part. Finally we can provide user-centered service with individual activity registration and classification for each user with our proposed system. Key words: life log system, spatiotemporal interface, activity analysis