Multiple multimodal mobile devices: Lessons learned from engineering lifelog solutions

For lifelogging, or the recording of one’s life history through digital means, to be successful, a range of separate multimodal mobile devices must be employed. These include smartphones such as the N95, the Microsoft SenseCam – a wearable passive photo capture device, or wearable biometric devices. Each collects a facet of the bigger picture, through, for example, personal digital photos, mobile messages and documents access history, but unfortunately, they operate independently and unaware of each other. This creates significant challenges for the practical application of these devices, the use and integration of their data and their operation by a user. In this chapter we discuss the software engineering challenges and their implications for individuals working on integration of data from multiple ubiquitous mobile devices drawing on our experiences working with such technology over the past several years for the development of integrated personal lifelogs. The chapter serves as an engineering guide to those considering working in the domain of lifelogging and more generally to those working with multiple multimodal devices and integration of their data

Combining brain-computer interfaces and assistive technologies: state-of-the-art and challenges

In recent years, new research has brought the field of EEG-based Brain-Computer Interfacing (BCI) out of its infancy and into a phase of relative maturity through many demonstrated prototypes such as brain-controlled wheelchairs, keyboards, and computer games. With this proof-of-concept phase in the past, the time is now ripe to focus on the development of practical BCI technologies that can be brought out of the lab and into real-world applications. In particular, we focus on the prospect of improving the lives of countless disabled individuals through a combination of BCI technology with existing assistive technologies (AT). In pursuit of more practical BCIs for use outside of the lab, in this paper, we identify four application areas where disabled individuals could greatly benefit from advancements in BCI technology, namely,“Communication and Control”, “Motor Substitution”, “Entertainment”, and “Motor Recovery”. We review the current state of the art and possible future developments, while discussing the main research issues in these four areas. In particular, we expect the most progress in the development of technologies such as hybrid BCI architectures, user-machine adaptation algorithms, the exploitation of users’ mental states for BCI reliability and confidence measures, the incorporation of principles in human-computer interaction (HCI) to improve BCI usability, and the development of novel BCI technology including better EEG devices

Imaging markers of disability in aquaporin-4 immunoglobulin G seropositive neuromyelitis optica: a graph theory study

Neuromyelitis optica spectrum disorders lack imaging biomarkers associated with disease course and supporting prognosis. This complex and heterogeneous set of disorders affects many regions of the central nervous system, including the spinal cord and visual pathway. Here, we use graph theory-based multimodal network analysis to investigate hypothesis-free mixed networks and associations between clinical disease with neuroimaging markers in 40 aquaporin-4-immunoglobulin G antibody seropositive patients (age = 48.16 ± 14.3 years, female:male = 36:4) and 31 healthy controls (age = 45.92 ± 13.3 years, female:male = 24:7). Magnetic resonance imaging measures included total brain and deep grey matter volumes, cortical thickness and spinal cord atrophy. Optical coherence tomography measures of the retina and clinical measures comprised of clinical attack types and expanded disability status scale were also utilized. For multimodal network analysis, all measures were introduced as nodes and tested for directed connectivity from clinical attack types and disease duration to systematic imaging and clinical disability measures. Analysis of variance, with group interactions, gave weights and significance for each nodal association (hyperedges). Connectivity matrices from 80% and 95% F-distribution networks were analyzed and revealed the number of combined attack types and disease duration as the most connected nodes, directly affecting changes in several regions of the central nervous system. Subsequent multivariable regression models, including interaction effects with clinical parameters, identified associations between decreased nucleus accumbens (β = −0.85, P = 0.021) and caudate nucleus (β = −0.61, P = 0.011) volumes with higher combined attack type count and longer disease duration, respectively. We also confirmed previously reported associations between spinal cord atrophy with increased number of clinical myelitis attacks. Age was the most important factor associated with normalized brain volume, pallidum volume, cortical thickness and the expanded disability status scale score. The identified imaging biomarker candidates warrant further investigation in larger-scale studies. Graph theory-based multimodal networks allow for connectivity and interaction analysis, where this method may be applied in other complex heterogeneous disease investigations with different outcome measures

WIMP astronomy and particle physics with liquid-noble and cryogenic direct-detection experiments

Once weakly-interacting massive particles (WIMPs) are unambiguously detected in direct-detection experiments, the challenge will be to determine what one may infer from the data. Here, I examine the prospects for reconstructing the local speed distribution of WIMPs in addition to WIMP particle-physics properties (mass, cross sections) from next-generation cryogenic and liquid-noble direct-detection experiments. I find that the common method of fixing the form of the velocity distribution when estimating constraints on WIMP mass and cross sections means losing out on the information on the speed distribution contained in the data and may lead to biases in the inferred values of the particle-physics parameters. I show that using a more general, empirical form of the speed distribution can lead to good constraints on the speed distribution. Moreover, one can use Bayesian model-selection criteria to determine if a theoretically-inspired functional form for the speed distribution (such as a Maxwell-Boltzmann distribution) fits better than an empirical model. The shape of the degeneracy between WIMP mass and cross sections and their offset from the true values of those parameters depends on the hypothesis for the speed distribution, which has significant implications for consistency checks between direct-detection and collider data. In addition, I find that the uncertainties on theoretical parameters depends sensitively on the upper end of the energy range used for WIMP searches. Better constraints on the WIMP particle-physics parameters and speed distribution are obtained if the WIMP search is extended to higher energy (~ 1 MeV).Comment: 25 pages, 27 figures, matches published versio

First impressions: A survey on vision-based apparent personality trait analysis

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Personality analysis has been widely studied in psychology, neuropsychology, and signal processing fields, among others. From the past few years, it also became an attractive research area in visual computing. From the computational point of view, by far speech and text have been the most considered cues of information for analyzing personality. However, recently there has been an increasing interest from the computer vision community in analyzing personality from visual data. Recent computer vision approaches are able to accurately analyze human faces, body postures and behaviors, and use these information to infer apparent personality traits. Because of the overwhelming research interest in this topic, and of the potential impact that this sort of methods could have in society, we present in this paper an up-to-date review of existing vision-based approaches for apparent personality trait recognition. We describe seminal and cutting edge works on the subject, discussing and comparing their distinctive features and limitations. Future venues of research in the field are identified and discussed. Furthermore, aspects on the subjectivity in data labeling/evaluation, as well as current datasets and challenges organized to push the research on the field are reviewed.Peer ReviewedPostprint (author's final draft

Translating novel findings of perceptual-motor codes into the neuro-rehabilitation of movement disorders

The bidirectional flow of perceptual and motor information has recently proven useful as rehabilitative tool for re-building motor memories. We analyzed how the visual-motor approach has been successfully applied in neurorehabilitation, leading to surprisingly rapid and effective improvements in action execution. We proposed that the contribution of multiple sensory channels during treatment enables individuals to predict and optimize motor behavior, having a greater effect than visual input alone. We explored how the state-of-the-art neuroscience techniques show direct evidence that employment of visual-motor approach leads to increased motor cortex excitability and synaptic and cortical map plasticity. This super-additive response to multimodal stimulation may maximize neural plasticity, potentiating the effect of conventional treatment, and will be a valuable approach when it comes to advances in innovative methodologies

Machine Analysis of Facial Expressions

No abstract