Organising and structuring a visual diary using visual interest point detectors

As wearable cameras become more popular, researchers are increasingly focusing on novel applications to manage the large volume of data these devices produce. One such application is the construction of a Visual Diary from an individual’s photographs. Microsoft’s SenseCam, a device designed to passively record a Visual Diary and cover a typical day of the user wearing the camera, is an example of one such device. The vast quantity of images generated by these devices means that the management and organisation of these collections is not a trivial matter. We believe wearable cameras, such as SenseCam, will become more popular in the future and the management of the volume of data generated by these devices is a key issue. Although there is a significant volume of work in the literature in the object detection and recognition and scene classification fields, there is little work in the area of setting detection. Furthermore, few authors have examined the issues involved in analysing extremely large image collections (like a Visual Diary) gathered over a long period of time. An algorithm developed for setting detection should be capable of clustering images captured at the same real world locations (e.g. in the dining room at home, in front of the computer in the office, in the park, etc.). This requires the selection and implementation of suitable methods to identify visually similar backgrounds in images using their visual features. We present a number of approaches to setting detection based on the extraction of visual interest point detectors from the images. We also analyse the performance of two of the most popular descriptors - Scale Invariant Feature Transform (SIFT) and Speeded Up Robust Features (SURF).We present an implementation of a Visual Diary application and evaluate its performance via a series of user experiments. Finally, we also outline some techniques to allow the Visual Diary to automatically detect new settings, to scale as the image collection continues to grow substantially over time, and to allow the user to generate a personalised summary of their data

Image metadata estimation using independent component analysis and regression

In this paper, we describe an approach to camera metadata estimation using regression based on Independent Component Analysis (ICA). Semantic scene classification of images using camera metadata related to capture conditions has had some success in the past. However, different makes and models of camera capture different types of metadata and this severely hampers the application of this kind of approach in real systems that consist of photos captured by many different users. We propose to address this issue by using regression to predict the missing metadata from observed data, thereby providing more complete (and hence more useful) metadata for the entire image corpus. The proposed approach uses an ICA based approach to regression

Keyframe detection in visual lifelogs

The SenseCam is a wearable camera that passively captures images. Therefore, it requires no conscious effort by a user in taking a photo. A Visual Diary from such a source could prove to be a valuable tool in assisting the elderly, individuals with neurodegenerative diseases, or other traumas. One issue with Visual Lifelogs is the large volume of image data generated. In previous work we spit a day's worth of images into more manageable segments, i.e. into distinct events or activities. However, each event coud stil consist of 80-100 images. thus, in this paper we propose a novel approach to selecting the key images within an event using a combination of MPEG-7 and Scale Invariant Feature Transform (SIFT) features

MyPlaces: detecting important settings in a visual diary

We describe a novel approach to identifying specific settings in large collections of passively captured images corresponding to a visual diary. An algorithm developed for setting detection should be capable of detecting images captured at the same real world locations (e.g. in the dining room at home, in front of the computer in the office, in the park, etc.). This requires the selection and implementation of suitable methods to identify visually similar backgrounds in images using their visual features. We use a Bag of Keypoints approach. This method is based on the sampling and subsequent vector quantization of multiple image patches. The image patches are sampled and described using Scale Invariant Feature Transform (SIFT) features. We compare two different classifiers, K Nearest Neighbour and Multiclass Linear Perceptron, and present results for classifying ten different settings across one week’s worth of images. Our results demonstrate that the method produces good classification accuracy even without exploiting geometric or context based information. We also describe an early prototype of a visual diary browser that integrates the classification results

The aceToolbox: low-level audiovisual feature extraction for retrieval and classification

In this paper we present an overview of a software platform that has been developed within the aceMedia project, termed the aceToolbox, that provides global and local lowlevel feature extraction from audio-visual content. The toolbox is based on the MPEG-7 eXperimental Model (XM), with extensions to provide descriptor extraction from arbitrarily shaped image segments, thereby supporting local descriptors reflecting real image content. We describe the architecture of the toolbox as well as providing an overview of the descriptors supported to date. We also briefly describe the segmentation algorithm provided. We then demonstrate the usefulness of the toolbox in the context of two different content processing scenarios: similarity-based retrieval in large collections and scene-level classification of still images

Exploiting context information to aid landmark detection in SenseCam images

In this paper, we describe an approach designed to exploit context information in order to aid the detection of landmark images from a large collection of photographs. The photographs were generated using Microsoft’s SenseCam, a device designed to passively record a visual diary and cover a typical day of the user wearing the camera. The proliferation of digital photos along with the associated problems of managing and organising these collections provide the background motivation for this work. We believe more ubiquitious cameras, such as SenseCam, will become the norm in the future and the management of the volume of data generated by such devices is a key issue. The goal of the work reported here is to use context information to assist in the detection of landmark images or sequences of images from the thousands of photos taken daily by SenseCam. We will achieve this by analysing the images using low-level MPEG-7 features along with metadata provided by SenseCam, followed by simple clustering to identify the landmark images

Perceptual plasticity in the peripheral visual field of older adults

Perceptual learning is an important mechanism in the human visual system, and can lead to long-lasting improvements across a broad range of perceptual tasks. In this study we demonstrated how perceptual learning can be applied to improve word recognition in the peripheral visual field of a sample of older individuals. We have shown that improvements in thresholds can be equalised across age, simply by increasing the number of training sessions available to older observers. Based on this initial finding we further sought to establish a protocol to induce improvements in reading ability for a sample of individuals with age-related macular disease (AMD). As a prelude to this work, we investigated the effects of crowding and fixation instability on similar tasks. Having suffered damage to their central vision, our target population (individuals with AMD) must use peripheral vision for daily viewing tasks. Peripheral vision is known to be highly susceptible to crowding, the influence of which has previously been shown to strengthen with age. We investigated the relationship between age and crowding on a letter recognition task, and found that (for this task) crowding was age in-variant, implying that this key inhibitor to peripheral visual perception should not have an inordinate influence on learning in our AMD sample. Our work on fixation stability also led to promising results. We demonstrated that our proxy for fixation instability (a dynamic target or dynamic fixation point) did not adversely affect letter recognition thresholds. Fixation instability is a common issue in AMD, but our data suggests that this may not adversely affect learning on our word recognition task. The final part of this work has been the implementation of a small study in which we trained a sample of individuals with AMD on our word recognition task. Significant improvements in thresholds were observed, though these did not quite reach the level of an age-matched normally sighted sample. Nonetheless, the trajectory of the learning curve suggests that further improvements would be possible with extended training sessions. Crucially, we also observed significant transfer of learning – from the trained word recognition task to an untrained sentence reading task (the MNRead Acuity chart). This is a key aspect of the study, as we are primarily interested in developing training protocols that lead to real-world improvements in visual ability. Improvements on MNRead scores are promising, and suggest that our approach may prove to be a useful starting point in the development of a robust therapeutic protocol

Diabetic Retinopathy Environment-Wide Association Study (EWAS) in NHANES 2005–2008

Several circulating biomarkers are reported to be associated with diabetic retinopathy (DR). However, their relative contributions to DR compared to known risk factors, such as hyperglycaemia, hypertension, and hyperlipidaemia, remain unclear. In this data driven study, we used novel models to evaluate the associations of over 400 laboratory parameters with DR compared to the established risk factors. Methods: we performed an environment-wide association study (EWAS) of laboratory parameters available in National Health and Nutrition Examination Survey (NHANES) 2007–2008 in individuals with diabetes with DR as the outcome (test set). We employed independent variable (feature) selection approaches, including parallelised univariate regression modelling, Principal Component Analysis (PCA), penalised regression, and RandomForest™. These models were replicated in NHANES 2005–2006 (replication set). Our test and replication sets consisted of 1025 and 637 individuals with available DR status and laboratory data respectively. Glycohemoglobin (HbA1c) was the strongest risk factor for DR. Our PCA-based approach produced a model that incorporated 18 principal components (PCs) that had an Area under the Curve (AUC) 0.796 (95% CI 0.761–0.832), while penalised regression identified a 9-feature model with 78.51% accuracy and AUC 0.74 (95% CI 0.72–0.77). RandomForest™ identified a 31-feature model with 78.4% accuracy and AUC 0.71 (95% CI 0.65–0.77). On grouping the selected variables in our RandomForest™, hyperglycaemia alone achieved AUC 0.72 (95% CI 0.68–0.76). The AUC increased to 0.84 (95% CI 0.78–0.9) when the model also included hypertension, hypercholesterolemia, haematocrit, renal, and liver function tests

Dementia care mapping in long-term care settings: a systematic review of the evidence

YesThis systematic review identifies and reports the extent and nature of evidence to support the use of Dementia Care Mapping as an intervention in care settings. The review was limited to studies that used Dementia Care Mapping as an intervention and included outcomes involving either care workers and/or people living with dementia. Searches were conducted in PubMed, Web of Knowledge, CINAHL, PsychINFO, EBSCO and Scopus and manually from identified articles reference lists. Studies published up to January 2017 were included. Initial screening of identified papers was based on abstracts read by one author; full-text papers were further evaluated by a second author. The quality of the identified papers was assessed independently by two authors using the Cochrane Risk of Bias Tool. A narrative synthesis of quantitative findings was conducted. We identified 6 papers fulfilling predefined criteria. Studies consist of recent, large scale, good quality trials that had some positive impacts upon care workers’ stress and burnout and benefit people with dementia in terms of agitated behaviours, neuropsychiatric symptoms, falls and quality of life. Available research provides preliminary evidence that Dementia Care Mapping may benefit care workers and people living with dementia in care settings. Future research should build on the successful studies to date and use other outcomes to better understand the benefits of this intervention

Perceptual plasticity in the peripheral visual field of older adults

Perceptual learning is an important mechanism in the human visual system, and can lead to long-lasting improvements across a broad range of perceptual tasks. In this study we demonstrated how perceptual learning can be applied to improve word recognition in the peripheral visual field of a sample of older individuals. We have shown that improvements in thresholds can be equalised across age, simply by increasing the number of training sessions available to older observers. Based on this initial finding we further sought to establish a protocol to induce improvements in reading ability for a sample of individuals with age-related macular disease (AMD). As a prelude to this work, we investigated the effects of crowding and fixation instability on similar tasks. Having suffered damage to their central vision, our target population (individuals with AMD) must use peripheral vision for daily viewing tasks. Peripheral vision is known to be highly susceptible to crowding, the influence of which has previously been shown to strengthen with age. We investigated the relationship between age and crowding on a letter recognition task, and found that (for this task) crowding was age in-variant, implying that this key inhibitor to peripheral visual perception should not have an inordinate influence on learning in our AMD sample. Our work on fixation stability also led to promising results. We demonstrated that our proxy for fixation instability (a dynamic target or dynamic fixation point) did not adversely affect letter recognition thresholds. Fixation instability is a common issue in AMD, but our data suggests that this may not adversely affect learning on our word recognition task. The final part of this work has been the implementation of a small study in which we trained a sample of individuals with AMD on our word recognition task. Significant improvements in thresholds were observed, though these did not quite reach the level of an age-matched normally sighted sample. Nonetheless, the trajectory of the learning curve suggests that further improvements would be possible with extended training sessions. Crucially, we also observed significant transfer of learning – from the trained word recognition task to an untrained sentence reading task (the MNRead Acuity chart). This is a key aspect of the study, as we are primarily interested in developing training protocols that lead to real-world improvements in visual ability. Improvements on MNRead scores are promising, and suggest that our approach may prove to be a useful starting point in the development of a robust therapeutic protocol