Understanding Social Context from Smartphone Sensing: Generalization Across Countries and Daily Life Moments

Understanding and longitudinally tracking the social context of people help in understanding their behavior and mental well-being better. Hence, instead of burdensome questionnaires, some studies used passive smartphone sensors to infer social context with machine learning models. However, the few studies that have been done up to date have focused on unique, situated contexts (i.e., when eating or drinking) in one or two countries, hence limiting the understanding of the inference in terms of generalization to (i) everyday life occasions and (ii) different countries. In this paper, we used a novel, large-scale, and multimodal smartphone sensing dataset with over 216K self-reports collected from over 580 participants in five countries (Mongolia, Italy, Denmark, UK, Paraguay), first to understand whether social context inference (i.e., alone or not) is feasible with sensor data, and then, to know how behavioral and country-level diversity affects the inference. We found that (i) sensor features from modalities such as activity, location, app usage, Bluetooth, and WiFi could be informative of social context; (ii) partially personalized multi-country models (trained and tested with data from all countries) and country-specific models (trained and tested within countries) achieved similar accuracies in the range of 80%-90%; and (iii) models do not generalize well to unseen countries regardless of geographic similarity

Active detection of age groups based on touch interaction

This paper studies user classification into children and adults according to their interaction with touchscreen devices. We analyse the performance of two sets of features derived from the Sigma-Lognormal theory of rapid human movements and global characterization of touchscreen interaction. We propose an active detection approach aimed to continuously monitorize the user patterns. The experimentation is conducted on a publicly available database with samples obtained from 89 children between 3 and 6 years old and 30 adults. We have used Support Vector Machines algorithm to classify the resulting features into age groups. The sets of features are fused at score level using data from smartphones and tablets. The results, with correct classification rates over 96%, show the discriminative ability of the proposed neuromotorinspired features to classify age groups according to the interaction with touch devices. In active detection setup, our method is able to identify a child using only 4 gestures in averageThis work was funded by the project CogniMetrics (TEC2015-70627-R) and Bio-Guard (Ayudas Fundación BBVA a Equipos de Investigación Científica 2017

Analysis of touch gestures for online child protection

AbstractThe growth of Internet and the pervasiveness of ICT have led to a radical change in social relationships. One of the drawbacks of this change is the exposure of individuals to threats during online activities. In this context, thetechno-regulationparadigm is inspiring new ways to safeguard legally interests by means of tools allowing to hamper breaches of law. In this paper, we focus on the exposure of individuals to specific online threats when interacting with smartphones. We propose a novel techno-regulatory approach exploiting machine learning techniques to provide safeguards against threats online. Specifically, we study a set of touch-based gestures to distinguish between underages or adults who is accessing a smartphone, and so to guarantee protection. To evaluate the proposed approach's effectiveness, we developed an Android app to build a dataset consisting of more than 9000 touch-gestures from 147 participants. We experimented bothsingle-viewandmulti-viewlearning techniques to find the best combination of touch-gestures able of distinguishing between adults and underages. Results show that the multi-view learning combining scrolls, swipes, and pinch-to-zoom gestures, achieves the best ROC AUC (0.92) and accuracy (88%) scores

Beyond mobile apps: a survey of technologies for mental well-being

Mental health problems are on the rise globally and strain national health systems worldwide. Mental disorders are closely associated with fear of stigma, structural barriers such as financial burden, and lack of available services and resources which often prohibit the delivery of frequent clinical advice and monitoring. Technologies for mental well-being exhibit a range of attractive properties, which facilitate the delivery of state-of-the-art clinical monitoring. This review article provides an overview of traditional techniques followed by their technological alternatives, sensing devices, behaviour changing tools, and feedback interfaces. The challenges presented by these technologies are then discussed with data collection, privacy, and battery life being some of the key issues which need to be carefully considered for the successful deployment of mental health toolkits. Finally, the opportunities this growing research area presents are discussed including the use of portable tangible interfaces combining sensing and feedback technologies. Capitalising on the data these ubiquitous devices can record, state of the art machine learning algorithms can lead to the development of robust clinical decision support tools towards diagnosis and improvement of mental well-being delivery in real-time

Exploring multimedia and interactive technologies

The goal of multimedia design strategies and innovation is to produce meaningful learning environments that relate to and build upon what the learner already knows and what the learner seeks. The multimedia tools used to achieve knowledge transfer should activate recall or prior knowledge and help the learner alter and encode new structures. Traditionally, multimedia has been localized to specific delivery systems and demographics based on the government, industry, or academic concentration. The presenter will explore the introduction of immersive telecommunications technologies, constructivist learning methodologies, and adult learning models to standardize networking and multimedia-based services and products capable of adapting to wired and wireless environments, different devices and conditions on a global scale

Educational E-book For Children With and Without Developmental Disorders

In the last decade, Autism Spectrum Disorder (ASD) prevalence rate has signifcantly increased, which consequently led to the expansion of research and expenditure in the feld,predominantly focusing on searching for the cause. In a typical classroom scenario, working with children with ASD very often requires 1:1 teacher to child ratio, which makes it very expensive and difcult to implement. Serious games have been utilised as a medium for teaching various developmental skills, such as social interaction, speech, motor skills development, emotion recognition, and other basic concepts. Designing serious games for ASD population difers from other games and even other serious games signifcantly. It requires a holistic approach with extensive knowledge and expertise from felds other than computer science, such as psychology, sociology and cognitive science. However, once harnessed correctly, such games can be used by children with ASD on their own time, with or without supervision and they can be educational. In addition, they can adjust the appropriate pace while at the same time providing feedback in form of reinforcement and correction. Applying the rules of science of learning and teaching, one can design games that are educational for all types of learners, including children with ASD. In this paper, two independent user studies have been conducted, demonstrating how serious gaming and e-learning principles can be harnessed in order to intervene, develop or strengthen pivotal developmental skills, like learning novel vocabulary, counting, identifying numbers and colours, and responding to inference questions. We have tested the educational e-book with children diagnosed with ASD and with typically developing children to assess skill acquisition in native language for children with ASD and in English, a foreign language, for typically developing children to demonstrate the educational aspect of the game for all types of learners. We showed that the same e-book in two languages can be used for teaching diferent types of learners through a fun and engaging medium

Tangible fidgeting interfaces for mental wellbeing recognition using deep learning applied to physiological sensor data

The momentary assessment of an individual's affective state is critical to the monitoring of mental wellbeing and the ability to instantly apply interventions. This thesis introduces the concept of tangible fidgeting interfaces for affective recognition from design and development through to evaluation. Tangible interfaces expand upon the affordance of familiar physical objects as the ability to touch and fidget may help to tap into individuals' psychological need to feel occupied and engaged. Embedding digital technologies within interfaces capitalises on motor and perceptual capabilities and allows for the direct manipulation of data, offering people the potential for new modes of interaction when experiencing mental wellbeing challenges. Tangible interfaces present an ideal opportunity to digitally enable physical fidgeting interactions along with physiological sensor monitoring to unobtrusively and comfortable measure non-visable changes in affective state. This opportunity initiated the investigation of factors that would bring about the designing of more effective intelligent solutions using participatory design techniques to engage people in designing solutions relevant to themselves. Adopting an artificial intelligence approach using physiological signals creates the possibility to quantify affect with high levels of accuracy. However, labelling is an indispensable stage of data pre-processing that is required before classification and can be extremely challenging with multi-model sensor data. New techniques are introduced for labelling at the point of collection coupled with a pilot study and a systematic performance comparison of five custom built labelling interfaces. When classifying labelled physiological sensor data, individual differences between people limit the generalisability of models. To address this challenge, a transfer learning approach has been developed that personalises affective models using few labelled samples. This approach to personalise models and improve cross-domain performance is completed on-device, automating the traditionally manual process, saving time and labour. Furthermore, monitoring trajectories over long periods of time inherits some critical limitations in relation to the size of the training dataset. This shortcoming may hinder the development of reliable and accurate machine learning models. A second framework has been developed to overcome the limitation of small training datasets using an image-encoding transfer learning approach. This research offers the first attempt at the development of tangible interfaces using artificial intelligence towards building a real-world continuous affect recognition system in addition to offering real-time feedback to perform as interventions. This exploration of affective interfaces has many potential applications to help improve quality of life for the wider population