Improving the classification of quantified self activities and behaviour using a Fisher kernel

Visual recording of everyday human activities and behaviour over the long term is now feasible and with the widespread use of wearable devices embedded with cameras this offers the potential to gain real insights into wearers’ activities and behaviour. To date we have concentrated on automatically detecting semantic concepts from within visual lifelogs yet identifying human activities from such lifelogged images or videos is still a major challenge if we are to use lifelogs to maximum benefit. In this paper, we propose an activity classification method from visual lifelogs based on Fisher kernels, which extract discriminative embeddings from Hidden Markov Models (HMMs) of occurrences of semantic concepts. By using the gradients as features, the resulting classifiers can better distinguish different activities and from that we can make inferences about human behaviour. Experiments show the effectiveness of this method in improving classification accuracy, especially when the semantic concepts are initially detected with low degrees of accuracy

Fast human activity recognition in lifelogging

This paper addresses the problem of fast Human Activity Recognition (HAR) in visual lifelogging. We identify the importance of visual features related to HAR and we specifically evaluate the HAR discrimination potential of Colour Histograms and Histogram of Oriented Gradients. In our evaluation we show that colour can be a low-cost and effective means of low-cost HAR when performing single-user classification. It is also noted that, while much more efficient, global image descriptors perform as well or better than local descriptors in our HAR experiments. We believe that both of these findings are due to the fact that a user’s lifelog is rich in reoccurring scenes and environments

Improving the Classification of Quantified Self Activities and Behaviour Using a Fisher Kernel

Abstract Visual recording of everyday human activities and behaviour over the long term is now feasible and with the widespread use of wearable devices embedded with cameras this offers the potential to gain real insights into wearers' activities and behaviour. To date we have concentrated on automatically detecting semantic concepts from within visual lifelogs yet identifying human activities from such lifelogged images or videos is still a major challenge if we are to use lifelogs to maximum benefit. In this paper, we propose an activity classification method from visual lifelogs based on Fisher kernels, which extract discriminative embeddings from Hidden Markov Models (HMMs) of occurrences of semantic concepts. By using the gradients as features, the resulting classifiers can better distinguish different activities and from that we can make inferences about human behaviour. Experiments show the effectiveness of this method in improving classification accuracy, especially when the semantic concepts are initially detected with low degrees of accuracy

Periodicity detection in lifelog data with missing and irregularly sampled data

Lifelogging is the ambient, continuous digital recording of a person’s everyday activities for a variety of possible applications. Much of the work to date in lifelogging has focused on developing sensors, capturing information, processing it into events and then supporting event-based access to the lifelog for applications like memory recall, behaviour analysis or similar. With the recent arrival of aggregating platforms such as Apple’s HealthKit, Microsoft’s HealthVault and Google’s Fit, we are now able to collect and aggregate data from lifelog sensors, to centralize the management of data and in particular to search for and detect patterns of usage for individuals and across populations. In this paper, we present a framework that detects both lowlevel and high-level periodicity in lifelog data, detecting hidden patterns of which users would not otherwise be aware. We detect periodicities of time series using a combination of correlograms and periodograms, using various signal processing algorithms. Periodicity detection in lifelogs is particularly challenging because the lifelog data itself is not always continuous and can have gaps as users may use their lifelog devices intermittingly. To illustrate that periodicity can be detected from such data, we apply periodicity detection on three lifelog datasets with varying levels of completeness and accuracy

The challenge of obtaining ground truth for GPS processing

The increasing use of GPS as a substitute or complement to conventional travel surveys has brought with it an increasing need for a reliable source of ground truth, i.e., information on the actual travel in which each respondent engaged, including the mode and purpose of such travel. Over the past decade or so, the main source that has been used for providing ground truth is the prompted recall survey. As the survey task has become less of a challenge, it has become evident that the prompted recall survey is subject to many of the same shortcomings as most selfadministered surveys, with reporting error, misunderstanding of what constitutes a trip, etc. In this paper, we review some of the common problems encountered in prompted recall surveys, especially those undertaken at some period of time after the GPS survey was undertaken, and using self-report methods. Following this, the paper describes a recent experiment in using life-logging cameras to record a person’s travel, together with GPS, thereby providing a new source of ground truth data that is not subject to processing issues of the GPS data, nor to self-report or other issues with prompted recall data

Continuous Health Interface Event Retrieval

Knowing the state of our health at every moment in time is critical for advances in health science. Using data obtained outside an episodic clinical setting is the first step towards building a continuous health estimation system. In this paper, we explore a system that allows users to combine events and data streams from different sources to retrieve complex biological events, such as cardiovascular volume overload. These complex events, which have been explored in biomedical literature and which we call interface events, have a direct causal impact on relevant biological systems. They are the interface through which the lifestyle events influence our health. We retrieve the interface events from existing events and data streams by encoding domain knowledge using an event operator language.Comment: ACM International Conference on Multimedia Retrieval 2020 (ICMR 2020), held in Dublin, Ireland from June 8-11, 202

Multiple Density Maps Information Fusion for Effectively Assessing Intensity Pattern of Lifelogging Physical Activity

Physical activity (PA) measurement is a crucial task in healthcare technology aimed at monitoring the progression and treatment of many chronic diseases. Traditional lifelogging PA measures require relatively high cost and can only be conducted in controlled or semi-controlled environments, though they exhibit remarkable precision of PA monitoring outcomes. Recent advancement of commercial wearable devices and smartphones for recording one’s lifelogging PA has popularized data capture in uncontrolled environments. However, due to diverse life patterns and heterogeneity of connected devices as well as the PA recognition accuracy, lifelogging PA data measured by wearable devices and mobile phones contains much uncertainty thereby limiting their adoption for healthcare studies. To improve the feasibility of PA tracking datasets from commercial wearable/mobile devices, this paper proposes a lifelogging PA intensity pattern decision making approach for lifelong PA measures. The method is to firstly remove some irregular uncertainties (IU) via an Ellipse fitting model, and then construct a series of monthly based hour-day density map images for representing PA intensity patterns with regular uncertainties (RU) on each month. Finally it explores Dempster-Shafer theory of evidence fusing information from these density map images for generating a decision making model of a final personal lifelogging PA intensity pattern. The approach has significantly reduced the uncertainties and incompleteness of datasets from third party devices. Two case studies on a mobile personalized healthcare platform MHA [1] connecting the mobile app Moves are carried out. The results indicate that the proposed approach can improve effectiveness of PA tracking devices or apps for various types of people who frequently use them as a healthcare indicator

Technologies that assess the location of physical activity and sedentary behavior: a systematic review

Background: The location in which physical activity and sedentary behavior are performed can provide valuable behavioral information, both in isolation and synergistically with other areas of physical activity and sedentary behavior research. Global positioning systems (GPS) have been used in physical activity research to identify outdoor location; however, while GPS can receive signals in certain indoor environments, it is not able to provide room- or subroom-level location. On average, adults spend a high proportion of their time indoors. A measure of indoor location would, therefore, provide valuable behavioral information. Objective: This systematic review sought to identify and critique technology which has been or could be used to assess the location of physical activity and sedentary behavior. Methods: To identify published research papers, four electronic databases were searched using key terms built around behavior, technology, and location. To be eligible for inclusion, papers were required to be published in English and describe a wearable or portable technology or device capable of measuring location. Searches were performed up to February 4, 2015. This was supplemented by backward and forward reference searching. In an attempt to include novel devices which may not yet have made their way into the published research, searches were also performed using three Internet search engines. Specialized software was used to download search results and thus mitigate the potential pitfalls of changing search algorithms. Results: A total of 188 research papers met the inclusion criteria. Global positioning systems were the most widely used location technology in the published research, followed by wearable cameras, and radio-frequency identification. Internet search engines identified 81 global positioning systems, 35 real-time locating systems, and 21 wearable cameras. Real-time locating systems determine the indoor location of a wearable tag via the known location of reference nodes. Although the type of reference node and location determination method varies between manufacturers, Wi-Fi appears to be the most popular method. Conclusions: The addition of location information to existing measures of physical activity and sedentary behavior will provide important behavioral information