Utilising wearable sensor technology to provide effective memory cues

We describe a wearable sensor technology that passively records "lifelog" images and sensor readings of a wearer's daily life. The focus of our work is not on aggregating, collecting or networking data as in the usual application of sensors in the Sensor Web, but rather on detecting events of interest to the wearer from a multi-sensor standalone device. These events of interest provide effective cues to allow people to more easily access their autobiographical memories. Early research indicates this technology may be potentially helpful for sufferers of neurodegenerative diseases such as Alzheimer's

Multimodal Wearable Intelligence for Dementia Care in Healthcare 4.0: A Survey

As a new revolution of Ubiquitous Computing and Internet of Things, multimodal wearable intelligence technique is rapidly becoming a new research topic in both academic and industrial fields. Owning to the rapid spread of wearable and mobile devices, this technique is evolving healthcare from traditional hub-based systems to more personalised healthcare systems. This trend is well-aligned with recent Healthcare 4.0 which is a continuous process of transforming the entire healthcare value chain to be preventive, precise, predictive and personalised, with significant benefits to elder care. But empowering the utility of multimodal wearable intelligence technique for elderly care like people with dementia is significantly challenging considering many issues, such as shortage of cost-effective wearable sensors, heterogeneity of wearable devices connected, high demand for interoperability, etc. Focusing on these challenges, this paper gives a systematic review of advanced multimodal wearable intelligence technologies for dementia care in Healthcare 4.0. One framework is proposed for reviewing the current research of wearable intelligence, and key enabling technologies, major applications, and successful case studies in dementia care, and finally points out future research trends and challenges in Healthcare 4.0

The smartphone as a platform for wearable cameras in health research

Background: The SenseCam, a small camera that is worn on the chest via a lanyard, is increasingly being deployed in a health research. However, the SenseCam is not yet in widespread use due to a variety of factors. It is proposed that the ubiquitous smartphone can provide a more accessible alternative to the SenseCam. Purpose: To perform an initial evaluation of the potential of the smartphone to become an alternative to the SenseCam. Methods: Adults were supplied with a smartphone running lifelogging software which they wore on a lanyard. Participants wore the smartphone for up to a day and the resulting lifelog data was both manually annotated and automatically analyzed for the presence of visual concepts. The results were compared to prior work using the SenseCam. Results: In total 166,000 smartphone photos were gathered from 47 individuals, along with associated sensor readings. The average time spent wearing the device across all users was 5 hours and 39 minutes (± 4 hours 11 minutes SD). A subset of 36,698 photos was selected for manual annotation by 5 researchers. Software analysis of these photos supports the automatic identification of activities to a similar level of accuracy as for SenseCam images in a previous study. Conclusions: Many aspects of the functionality of a SenseCam can largely be replicated, and in some cases enhanced, by the ubiquitous smartphone platform. This makes the smartphone a good candidate for a new generation of wearable sensing devices in health research, due to its widespread use across many populations. It is envisioned that the smartphone will provide a compelling alternative to the dedicated SenseCam hardware for a number of users and application areas. This will be achieved by integrating new types of sensor data, leveraging the smartphone’s real-time connectivity and rich user interface, and providing support for a range of relatively sophisticated applications

Augmented reality and scene examination

The research presented in this thesis explores the impact of Augmented Reality on human performance, and compares this technology with Virtual Reality using a head-mounted video-feed for a variety of tasks that relate to scene examination. The motivation for the work was the question of whether Augmented Reality could provide a vehicle for training in crime scene investigation. The Augmented Reality application was developed using fiducial markers in the Windows Presentation Foundation, running on a wearable computer platform; Virtual Reality was developed using the Crytek game engine to present a photo-realistic 3D environment; and a video-feed was provided through head-mounted webcam. All media were presented through head-mounted displays of similar resolution to provide the sole source of visual information to participants in the experiments. The experiments were designed to increase the amount of mobility required to conduct the search task, i.e., from rotation in the horizontal or vertical plane through to movement around a room. In each experiment, participants were required to find objects and subsequently recall their location. It is concluded that human performance is affected not merely via the medium through which the world is perceived but moreover, the constraints governing how movement in the world is controlled

How much spatial information is lost in the sensory substitution process? Comparing visual, tactile, and auditory approaches

Sensory substitution devices (SSDs) can convey visuospatial information through spatialised auditory or tactile stimulation using wearable technology. However, the level of information loss associated with this transformation is unknown. In this study novice users discriminated the location of two objects at 1.2m using devices that transformed a 16x 8 depth map into spatially distributed patterns of light, sound, or touch on the abdomen. Results showed that through active sensing, participants could discriminate the vertical position of objects to a visual angle of 1°, 14°, and 21°, and their distance to 2cm, 8cm, and 29cm using these visual, auditory, and haptic SSDs respectively. Visual SSDs significantly outperformed auditory and tactile SSDs on vertical localisation, whereas for depth perception, all devices significantly differed from one another (visual > auditory > haptic). Our findings highlight the high level of acuity possible for SSDs even with low spatial resolutions (e.g. 16 8) and quantify the level of information loss attributable to this transformation for the SSD user. Finally, we discuss ways of closing this ‘modality gap’ found in SSDs and conclude that this process is best benchmarked against performance with SSDs that return to their primary modality (e.g. visuospatial into visual)

Towards disappearing user interfaces for ubiquitous computing: human enhancement from sixth sense to super senses

The enhancement of human senses electronically is possible when pervasive computers interact unnoticeably with humans in Ubiquitous Computing. The design of computer user interfaces towards “disappearing” forces the interaction with humans using a content rather than a menu driven approach, thus the emerging requirement for huge number of non-technical users interfacing intuitively with billions of computers in the Internet of Things is met. Learning to use particular applications in Ubiquitous Computing is either too slow or sometimes impossible so the design of user interfaces must be naturally enough to facilitate intuitive human behaviours. Although humans from different racial, cultural and ethnic backgrounds own the same physiological sensory system, the perception to the same stimuli outside the human bodies can be different. A novel taxonomy for Disappearing User Interfaces (DUIs) to stimulate human senses and to capture human responses is proposed. Furthermore, applications of DUIs are reviewed. DUIs with sensor and data fusion to simulate the Sixth Sense is explored. Enhancement of human senses through DUIs and Context Awareness is discussed as the groundwork enabling smarter wearable devices for interfacing with human emotional memories

Designing and evaluating a user interface for continous embedded lifelogging based on physical context

PhD ThesisAn increase in both personal information and storage capacity has encouraged people to store and archive their life experience in multimedia formats. The usefulness of such large amounts of data will remain inadequate without the development of both retrieval techniques and interfaces that help people access and navigate their personal collections. The research described in this thesis investigates lifelogging technology from the perspective of the psychology of memory and human-computer interaction. The research described seeks to increase my understanding of what data can trigger memories and how I might use this insight to retrieve past life experiences in interfaces to lifelogging technology. The review of memory and previous research on lifelogging technology allows and support me to establish a clear understanding of how memory works and design novel and effective memory cues; whilst at the same time I critiqued existing lifelogging systems and approaches to retrieving memories of past actions and activities. In the initial experiments I evaluated the design and implementation of a prototype which exposed numerous problems both in the visualisation of data and usability. These findings informed the design of novel lifelogging prototype to facilitate retrieval. I assessed the second prototype and determined how an improved system supported access and retrieval of users’ past life experiences, in particular, how users group their data into events, how they interact with their data, and the classes of memories that it supported. In this doctoral thesis I found that visualizing the movements of users’ hands and bodies facilitated grouping activities into events when combined with the photos and other data captured at the same time. In addition, the movements of the user's hand and body and the movements of some objects can promote an activity recognition or support user detection and grouping of them into events. Furthermore, the ability to search for specific movements significantly reduced the amount of time that it took to retrieve data related to specific events. I revealed three major strategies that users followed to understand the combined data: skimming sequences, cross sensor jumping and continued scanning

Devices and Data Workflow in COPD Wearable Remote Patient Monitoring: A Systematic Review

Background: With global increase in Chronic Obstructive Pulmonary Disease (COPD) prevalence and mortality rates, and socioeconomical burden continuing to rise, current disease management strategies appear inadequate, paving the way for technological solutions, namely remote patient monitoring (RPM), adoption considering its acute disease events management benefit. One RPM’s category stands out, wearable devices, due to its availability and apparent ease of use. Objectives: To assess the current market and interventional solutions regarding wearable devices in the remote monitoring of COPD patients through a systematic review design from a device composition, data workflow, and collected parameters description standpoint. Methods: A systematic review was conducted to identify wearable device trends in this population through the development of a comprehensive search strategy, searching beyond the mainstream databases, and aggregating diverse information found regarding the same device. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines were followed, and quality appraisal of identified studies was performed using the Critical Appraisal Skills Programme (CASP) quality appraisal checklists. Results: The review resulted on the identification of 1590 references, of which a final 79 were included. 56 wearable devices were analysed, with the slight majority belonging to the wellness devices class. Substantial device heterogeneity was identified regarding device composition type and wearing location, and data workflow regarding 4 considered components. Clinical monitoring devices are starting to gain relevance in the market and slightly over a third, aim to assist COPD patients and healthcare professionals in exacerbation prediction. Compliance with validated recommendations is still lacking, with no devices assessing the totality of recommended vital signs. Conclusions: The identified heterogeneity, despite expected considering the relative novelty of wearable devices, alerts for the need to regulate the development and research of these technologies, specially from a structural and data collection and transmission standpoints.Introdução: Com o aumento global das taxas de prevalência e mortalidade da Doença Pulmonar Obstrutiva Crónica (DPOC) e o seu impacto socioeconómico, as atuais estratégias de gestão da doença parecem inadequadas, abrindo caminho para soluções tecnológicas, nomeadamente para a adoção da monitorização remota, tendo em conta o seu benefício na gestão de exacerbações de doenças crónicas. Dentro destaca-se uma categoria, os dispositivos wearable, pela sua disponibilidade e aparente facilidade de uso. Objetivos: Avaliar as soluções existentes, tanto no mercado, como na área de investigação, relativas a dispositivos wearable utilizados na monitorização remota de pacientes com DPOC através de uma revisão sistemática, do ponto de vista da composição do dispositivo, fluxo de dados e descrição dos parâmetros coletados. Métodos: Uma revisão sistemática foi realizada para identificar tendências destes dispositivos, através do desenvolvimento de uma estratégia de pesquisa abrangente, procurando pesquisar para além das databases convencionais e agregar diversas informações encontradas sobre o mesmo dispositivo. Para tal, foram seguidas as diretrizes PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), e a avaliação da qualidade dos estudos identificados foi realizada utilizando a ferramenta CASP (Critical Appraisal Skills Programme). Resultados: A revisão resultou na identificação de 1590 referências, das quais 79 foram incluídas. Foram analisados 56 dispositivos wearable, com a ligeira maioria a pertencer à classe de dispositivos de wellness. Foi identificada heterogeneidade substancial nos dispositivos em relação à sua composição, local de uso e ao fluxo de dados em relação a 4 componentes considerados. Os dispositivos de monitorização clínica já evidenciam alguma relevância no mercado e, pouco mais de um terço, visam auxiliar pacientes com DPOC e profissionais de saúde na previsão de exacerbações. Ainda assim, é notória a falta do cumprimento das recomendações validadas, não estando disponíveis dispositivos que avaliem a totalidade dos sinais vitais recomendados. Conclusão: A heterogeneidade identificada, apesar de esperada face à relativa novidade dos dispositivos wearable, alerta para a necessidade de regulamentação do desenvolvimento e investigação destas tecnologias, especialmente do ponto de vista estrutural e de recolha e transmissão de dados

LifeLogging: personal big data

We have recently observed a convergence of technologies to foster the emergence of lifelogging as a mainstream activity. Computer storage has become significantly cheaper, and advancements in sensing technology allows for the efficient sensing of personal activities, locations and the environment. This is best seen in the growing popularity of the quantified self movement, in which life activities are tracked using wearable sensors in the hope of better understanding human performance in a variety of tasks. This review aims to provide a comprehensive summary of lifelogging, to cover its research history, current technologies, and applications. Thus far, most of the lifelogging research has focused predominantly on visual lifelogging in order to capture life details of life activities, hence we maintain this focus in this review. However, we also reflect on the challenges lifelogging poses to an information retrieval scientist. This review is a suitable reference for those seeking a information retrieval scientist’s perspective on lifelogging and the quantified self

The development and applications of serious games in the public services: defence and health

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London.The latest advances of Virtual Reality technologies and three-dimensional graphics, as well as the developments in Gaming Technologies in the recent years, have stemmed the proliferation of Serious Games in a broader spectrum of research applications. Among the most popular areas of application are public services such as Defence and Health, where digital technologies realise new challenges and opportunities for research and development of Serious Games and for a variety of contexts. As with all games, the user engagement is elevated and apart from the entertaining aspect, Serious Games serve as a novel and promising alternative experience to knowledge transfer. Furthermore, Serious Games bring to the end user and the overall society a series of attractive benefits. These benefits include safety, cost-effectiveness, increased motivation and personalisation. Hence, this Thesis aims to investigate novel approaches of developing Serious Games that utilise the recent advances of Virtual Reality and Gaming Technology and facilitate the aforementioned benefits. The process of design and development of the novel tools and applications follow an iterative manner and are driven by the review of the available literature as well as end-user feedbackEPSRC (Engineering and Physical Sciences Research Council ) , MOD (UK), NHS (UK