SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured social settings (e.g., cocktail party ) is gratifying due to the wealth of information available at the group (mining social networks) and individual (recognizing native behavioral and personality traits) levels. However, analyzing social scenes involving FCGs is also highly challenging due to the difficulty in extracting behavioral cues such as target locations, their speaking activity and head/body pose due to crowdedness and presence of extreme occlusions. To this end, we propose SALSA, a novel dataset facilitating multimodal and Synergetic sociAL Scene Analysis, and make two main contributions to research on automated social interaction analysis: (1) SALSA records social interactions among 18 participants in a natural, indoor environment for over 60 minutes, under the poster presentation and cocktail party contexts presenting difficulties in the form of low-resolution images, lighting variations, numerous occlusions, reverberations and interfering sound sources; (2) To alleviate these problems we facilitate multimodal analysis by recording the social interplay using four static surveillance cameras and sociometric badges worn by each participant, comprising the microphone, accelerometer, bluetooth and infrared sensors. In addition to raw data, we also provide annotations concerning individuals' personality as well as their position, head, body orientation and F-formation information over the entire event duration. Through extensive experiments with state-of-the-art approaches, we show (a) the limitations of current methods and (b) how the recorded multiple cues synergetically aid automatic analysis of social interactions. SALSA is available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure

Personalised Ambient Monitoring (PAM) for People with Bipolar Disorder

This paper presents the architecture and preliminary trial results of a monitoring system for patients with bipolar disorder containing environmental and wearable sensors

Exploring the Landscape of Ubiquitous In-home Health Monitoring: A Comprehensive Survey

Ubiquitous in-home health monitoring systems have become popular in recent years due to the rise of digital health technologies and the growing demand for remote health monitoring. These systems enable individuals to increase their independence by allowing them to monitor their health from the home and by allowing more control over their well-being. In this study, we perform a comprehensive survey on this topic by reviewing a large number of literature in the area. We investigate these systems from various aspects, namely sensing technologies, communication technologies, intelligent and computing systems, and application areas. Specifically, we provide an overview of in-home health monitoring systems and identify their main components. We then present each component and discuss its role within in-home health monitoring systems. In addition, we provide an overview of the practical use of ubiquitous technologies in the home for health monitoring. Finally, we identify the main challenges and limitations based on the existing literature and provide eight recommendations for potential future research directions toward the development of in-home health monitoring systems. We conclude that despite extensive research on various components needed for the development of effective in-home health monitoring systems, the development of effective in-home health monitoring systems still requires further investigation.Comment: 35 pages, 5 figure

Creating human digital memories with the aid of pervasive mobile devices

The abundance of mobile and sensing devices, within our environment, has led to a society in which any object, embedded with sensors, is capable of providing us with information. A human digital memory, created with the data from these pervasive devices, produces a more dynamic and data rich memory. Information such as how you felt, where you were and the context of the environment can be established. This paper presents the DigMem system, which utilizes distributed mobile services, linked data and machine learning to create such memories. Along with the design of the system, a prototype has also been developed, and two case studies have been undertaken, which successfully create memories. As well as demonstrating how memories are created, a key concern in human digital memory research relates to the amount of data that is generated and stored. In particular, searching this set of big data is a key challenge. In response to this, the paper evaluates the use of machine learning algorithms, as an alternative to SPARQL, and treats searching as a classification problem. In particular, supervised machine learning algorithms are used to find information in semantic annotations, based on probabilistic reasoning. Our approach produces good results with 100% sensitivity, 93% specificity, 93% positive predicted value, 100% negative predicted value, and an overall accuracy of 97%

Creating human digital memories with the aid of pervasive mobile devices

The abundance of mobile and sensing devices, within our environment, has led to a society in which any object, embedded with sensors, is capable of providing us with information. A human digital memory, created with the data from these pervasive devices, produces a more dynamic and data rich memory. Information such as how you felt, where you were and the context of the environment can be established. This paper presents the DigMem system, which utilizes distributed mobile services, linked data and machine learning to create such memories. Along with the design of the system, a prototype has also been developed, and two case studies have been undertaken, which successfully create memories. As well as demonstrating how memories are created, a key concern in human digital memory research relates to the amount of data that is generated and stored. In particular, searching this set of big data is a key challenge. In response to this, the paper evaluates the use of machine learning algorithms, as an alternative to SPARQL, and treats searching as a classification problem. In particular, supervised machine learning algorithms are used to find information in semantic annotations, based on probabilistic reasoning. Our approach produces good results with 100% sensitivity, 93% specificity, 93% positive predicted value, 100% negative predicted value, and an overall accuracy of 97%. Crown Copyright © 2013 Published by Elsevier B.V. All rights reserved

Mobility is the Message: Experiments with Mobile Media Sharing

This thesis explores new mobile media sharing applications by building, deploying, and studying their use. While we share media in many different ways both on the web and on mobile phones, there are few ways of sharing media with people physically near us. Studied were three designed and built systems: Push!Music, Columbus, and Portrait Catalog, as well as a fourth commercially available system – Foursquare. This thesis offers four contributions: First, it explores the design space of co-present media sharing of four test systems. Second, through user studies of these systems it reports on how these come to be used. Third, it explores new ways of conducting trials as the technical mobile landscape has changed. Last, we look at how the technical solutions demonstrate different lines of thinking from how similar solutions might look today. Through a Human-Computer Interaction methodology of design, build, and study, we look at systems through the eyes of embodied interaction and examine how the systems come to be in use. Using Goffman’s understanding of social order, we see how these mobile media sharing systems allow people to actively present themselves through these media. In turn, using McLuhan’s way of understanding media, we reflect on how these new systems enable a new type of medium distinct from the web centric media, and how this relates directly to mobility. While media sharing is something that takes place everywhere in western society, it is still tied to the way media is shared through computers. Although often mobile, they do not consider the mobile settings. The systems in this thesis treat mobility as an opportunity for design. It is still left to see how this mobile media sharing will come to present itself in people’s everyday life, and when it does, how we will come to understand it and how it will transform society as a medium distinct from those before. This thesis gives a glimpse at what this future will look like

Enhancing the museum experience with a sustainable solution based on contextual information obtained from an on-line analysis of users’ behaviour

Human computer interaction has evolved in the last years in order to enhance users’ experiences and provide more intuitive and usable systems. A major leap through in this scenario is obtained by embedding, in the physical environment, sensors capable of detecting and processing users’ context (position, pose, gaze, ...). Feeded by the so collected information flows, user interface paradigms may shift from stereotyped gestures on physical devices, to more direct and intuitive ones that reduce the semantic gap between the action and the corresponding system reaction or even anticipate the user’s needs, thus limiting the overall learning effort and increasing user satisfaction. In order to make this process effective, the context of the user (i.e. where s/he is, what is s/he doing, who s/he is, what are her/his preferences and also actual perception and needs) must be properly understood. While collecting data on some aspects can be easy, interpreting them all in a meaningful way in order to improve the overall user experience is much harder. This is more evident when we consider informal learning environments like museums, i.e. places that are designed to elicit visitor response towards the artifacts on display and the cultural themes proposed. In such a situation, in fact, the system should adapt to the attention paid by the user choosing the appropriate content for the user’s purposes, presenting an intuitive interface to navigate it. My research goal is focused on collecting, in a simple,unobtrusive, and sustainable way, contextual information about the visitors with the purpose of creating more engaging and personalized experiences

Advanced Internet of Things for Personalised Healthcare System: A Survey

As a new revolution of the Internet, Internet of Things (IoT) is rapidly gaining ground as a new research topic in many academic and industrial disciplines, especially in healthcare. Remarkably, due to the rapid proliferation of wearable devices and smartphone, the Internet of Things enabled technology is evolving healthcare from conventional hub based system to more personalised healthcare system (PHS). However, empowering the utility of advanced IoT technology in PHS is still significantly challenging in the area considering many issues, like shortage of cost-effective and accurate smart medical sensors, unstandardized IoT system architectures, heterogeneity of connected wearable devices, multi-dimensionality of data generated and high demand for interoperability. In an effect to understand advance of IoT technologies in PHS, this paper will give a systematic review on advanced IoT enabled PHS. It will review the current research of IoT enabled PHS, and key enabling technologies, major IoT enabled applications and successful case studies in healthcare, and finally point out future research trends and challenges