Segmenting and summarizing general events in a long-term lifelog

Lifelogging aims to capture a person’s life experiences using digital devices. When captured over an extended period of time a lifelog can potentially contain millions of files from various sources in a range of formats. For lifelogs containing such massive numbers of items, we believe it is important to group them into meaningful sets and summarize them, so that users can search and browse their lifelog data efficiently. Existing studies have explored the segmentation of continuously captured images over short periods of at most a few days into small groups of “events” (episodes). Yet, for long-term lifelogs, higher levels of abstraction are desirable due to the very large number of “events” which will occur over an extended period. We aim to segment a long-term lifelog at the level of general events which typically extend beyond a daily boundary, and to select summary information to represent these events. We describe our current work on higher level segmentation and summary information extraction for long term life logs and report a preliminary pilot study on a real long-term lifelog collection

TimelineQA: A Benchmark for Question Answering over Timelines

Lifelogs are descriptions of experiences that a person had during their life. Lifelogs are created by fusing data from the multitude of digital services, such as online photos, maps, shopping and content streaming services. Question answering over lifelogs can offer personal assistants a critical resource when they try to provide advice in context. However, obtaining answers to questions over lifelogs is beyond the current state of the art of question answering techniques for a variety of reasons, the most pronounced of which is that lifelogs combine free text with some degree of structure such as temporal and geographical information. We create and publicly release TimelineQA1, a benchmark for accelerating progress on querying lifelogs. TimelineQA generates lifelogs of imaginary people. The episodes in the lifelog range from major life episodes such as high school graduation to those that occur on a daily basis such as going for a run. We describe a set of experiments on TimelineQA with several state-of-the-art QA models. Our experiments reveal that for atomic queries, an extractive QA system significantly out-performs a state-of-the-art retrieval-augmented QA system. For multi-hop queries involving aggregates, we show that the best result is obtained with a state-of-the-art table QA technique, assuming the ground truth set of episodes for deriving the answer is available

The narrative self, distributed memory, and evocative objects

In this article, I outline various ways in which artifacts are interwoven with autobiographical memory systems and conceptualize what this implies for the self. I first sketch the narrative approach to the self, arguing that who we are as persons is essentially our (unfolding) life story, which, in turn, determines our present beliefs and desires, but also directs our future goals and actions. I then argue that our autobiographical memory is partly anchored in our embodied interactions with an ecology of artifacts in our environment. Lifelogs, photos, videos, journals, diaries, souvenirs, jewelry, books, works of art, and many other meaningful objects trigger and sometimes constitute emotionally-laden autobiographical memories. Autobiographical memory is thus distributed across embodied agents and various environmental structures. To defend this claim, I draw on and integrate distributed cognition theory and empirical research in human-technology interaction. Based on this, I conclude that the self is neither defined by psychological states realized by the brain nor by biological states realized by the organism, but should be seen as a distributed and relational construct

LifeLogging: personal big data

We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging in order to capture life details of life activities, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses to an information retrieval scientist. This review is a suitable reference for those seeking a information retrieval scientist’s perspective on lifelogging and the quantified self

Semantics-based selection of everyday concepts in visual lifelogging

Concept-based indexing, based on identifying various semantic concepts appearing in multimedia, is an attractive option for multimedia retrieval and much research tries to bridge the semantic gap between the media’s low-level features and high-level semantics. Research into concept-based multimedia retrieval has generally focused on detecting concepts from high quality media such as broadcast TV or movies, but it is not well addressed in other domains like lifelogging where the original data is captured with poorer quality. We argue that in noisy domains such as lifelogging, the management of data needs to include semantic reasoning in order to deduce a set of concepts to represent lifelog content for applications like searching, browsing or summarisation. Using semantic concepts to manage lifelog data relies on the fusion of automatically-detected concepts to provide a better understanding of the lifelog data. In this paper, we investigate the selection of semantic concepts for lifelogging which includes reasoning on semantic networks using a density-based approach. In a series of experiments we compare different semantic reasoning approaches and the experimental evaluations we report on lifelog data show the efficacy of our approach

Overview of NTCIR-13 Lifelog-2 Task

In this paper we review the NTCIR13-Lifelog core task, which ran at NTCIR-13. We outline the test collection employed, along with the tasks, the submissions and the findings from this pilot task. We finish by suggesting future plans for the task