Venturing into the labyrinth: the information retrieval challenge of human digital memories

Advances in digital capture and storage technologies mean that it is now possible to capture and store one’s entire life experiences in a Human Digital Memory (HDM). However, these vast personal archives are of little benefit if an individual cannot locate and retrieve significant items from them. While potentially offering exciting opportunities to support a user in their activities by providing access to information stored from previous experiences, we believe that the features of HDM datasets present new research challenges for information retrieval which must be addressed if these possibilities are to be realised. Specifically we postulate that effective retrieval from HDMs must exploit the rich sources of context data which can be captured and associated with items stored within them. User’s memories of experiences stored within their memory archive will often be linked to these context features. We suggest how such contextual metadata can be exploited within the retrieval process

An exploration of the utility of GSR in locating events from personal lifelogs for reflection

Digital personal lifelogs (PLs) enable many artifacts from a person’s life to be automatically stored in a digital archive. These data sets can contain a wealth of potentially valuable information describing events from an individual’s life. A key challenge for lifelog technologies is how to develop scenarios and applications which enable people to interact with these vast heterogeneous data sources in a meaningful way. One of the areas where individuals can gain from interacting with lifelog records of their life is in the process of self reflection. To date little attention has been given to applications which automatically extract content from lifelogs to support self reflection using lifelog content. One of the significant issues with reflection from lifelogs is discerning material which may be of interest in reflection from among the huge amount of available data. One way of determining the user’s engagement with their situation is measuring their biometric response associated with their arousal level. Specifically it is known that an individual’s galvanic skin response (GSR) can vary with their level of arousal. We hypothesize that situations of marked GSR variation are likely to be more significant for self reflection than other moments. We present an initial investigation, using 3 subjects’ lifelogs, of the utility of lifelog items with marked GSR for self reflection. Our results indicate that GSR records may serve as a good enabling technology for applications supporting self reflection and awareness

Examining the utility of affective response in search of personal lifelogs

Personal lifelog archives contain digital records captured from an individual’s daily life, for example emails, documents edited, webpages downloaded and photographs taken. While capturing this information is becoming increasingly easy, subsequently locating interesting items from within these archives is a significant challenge. One potential source of information to identify items of importance to an individual is their affective state during the capture of the information. The strength of an individual’s affective response to their current situation can often be gauged from their physiological response. For this study we explored the utility of the following biometric features to indicate significant items: galvanic skin response (GSR), heart rate (HR) and skin temperature (ST). Significant or important events tend to raise an individual’s arousal level, causing a measurable biometric response. We examined the utility of using biometric response to identify significant items and for re-ranking traditional information retrieval (IR) result sets. Results obtained indicate that skin temperature is most useful for extracting interesting items from personal archives containing passively captured images, computer activity and SMS messages

Multiple multimodal mobile devices: Lessons learned from engineering lifelog solutions

For lifelogging, or the recording of one’s life history through digital means, to be successful, a range of separate multimodal mobile devices must be employed. These include smartphones such as the N95, the Microsoft SenseCam – a wearable passive photo capture device, or wearable biometric devices. Each collects a facet of the bigger picture, through, for example, personal digital photos, mobile messages and documents access history, but unfortunately, they operate independently and unaware of each other. This creates significant challenges for the practical application of these devices, the use and integration of their data and their operation by a user. In this chapter we discuss the software engineering challenges and their implications for individuals working on integration of data from multiple ubiquitous mobile devices drawing on our experiences working with such technology over the past several years for the development of integrated personal lifelogs. The chapter serves as an engineering guide to those considering working in the domain of lifelogging and more generally to those working with multiple multimodal devices and integration of their data

The role of places and spaces in lifelog retrieval

Finding relevant interesting items when searching or browsing within a large multi-modal personal lifelog archive is a significant challenge. The use of contextual cues to filter the collection and aid in the determination of relevant content is often suggested as means to address such challenges. This work presents an exploration of the various locations, garnered through context logging, several participants engaged in during personal information access over a 15 month period. We investigate the implications of the varying data accessed across multiple locations for context-based retrieval from such collections. Our analysis highlights that a large number of spaces and places may be used for information access, but high volume of content is accessed in few

Considering subjects and scenarios in large-scale user-centered evaluation of a multilingual multimodal medical search system

Medical search applications can be required to service the differing information needs of multiple classes of users with varying medical knowledge levels, and language skills, as well as varying querying behaviours. The precise nature of these users' needs has to be understood to develop effective applications. Evaluation of developed search applications requires creation of holistic user-centred evaluation approaches which allow for comprehensive evaluation while being mindful of the diversity of users

Applying contextual memory cues for retrieval from personal information archives

Advances in digital technologies for information capture combined with massive increases in the capacity of digital storage media mean that it is now possible to capture and store one’s entire life experiences in a Human Digital Memory (HDM). Information can be captured from a myriad of personal information devices including desktop computers, PDAs, digital cameras, video and audio recorders, and various sensors, including GPS, Bluetooth, and biometric devices. These diverse collections of personal information are potentially very valuable, but will only be so if significant information can be reliably retrieved from them. HDMs differ from traditional document collections for which existing search technologies have been developed since users may have poor recollection of contents or even the existence of stored items. Additionally HDM data is highly heterogeneous and unstructured, making it difficult to form search queries. We believe that a Personal Information Management (PIM) system which exploits the context of information capture, and potentially of earlier refinding, can be valuable in effective retrieval from an HDM. We report an investigation into how individuals perform searches of their personal information, and use the outcome of this study to develop an information retrieval (IR) framework for HDM search incorporating the context of document capture. We then describe the creation of a pilot HDM test collection, and initial experiments in retrieval from this collection. Results from these experiments indicate that use of context data can be significantly beneficial to increasing the efficient retrieval of partially recalled items from an HDM

Memory support for desktop search

The user's memory plays a very important role in desktop search. A search query with insufficiently or inaccurately recalled information may make the search dramatically less effective. In this paper, we discuss three approaches to support user’s memory during desktop search. These include extended types of well remembered search options, the use of past search queries and results, and search from similar items. We will also introduce our search system which incorporates these features

DCU@TRECMed 2012: Using ad-hoc baselines for domain-specific retrieval

This paper describes the first participation of DCU in the TREC Medical Records Track (TRECMed). We performed some initial experiments on the 2011 TRECMed data based on the BM25 retrieval model. Surprisingly, we found that the standard BM25 model with default parameters, performs comparable to the best automatic runs submitted to TRECMed 2011 and would have resulted in rank four out of 29 participating groups. We expected that some form of domain adaptation would increase performance. However, results on the 2011 data proved otherwise: concept-based query expansion decreased performance, and filtering and reranking by term proximity also decreased performance slightly. We submitted four runs based on the BM25 retrieval model to TRECMed 2012 using standard BM25, standard query expansion, result filtering, and concept-based query expansion. Official results for 2012 confirm that domain-specific knowledge does not increase performance compared to the BM25 baseline as applied by us

A study of remembered context for information access from personal digital archives

Retrieval from personal archives (or Human Digital Memories (HDMs)) is set to become a significant challenge in information retrieval (IR) research. These archives are unique in that the items in them are personal to the owner and as such the owner may have personal memories associated with the items. It is recognized that the harnessing of an individual’s memories about HDM items can be used as context data (such as user location at the time of item access) to aid retrieval. We present a pilot study, using one subject’s HDM, of remembered context data and its utility in retrieval. Our results explore the types of context data best remembered for different item types and categories over time and show that context appears to become a more important factor in effective HDM IR over time as the subject’s recall of contents declines