SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured social settings (e.g., cocktail party ) is gratifying due to the wealth of information available at the group (mining social networks) and individual (recognizing native behavioral and personality traits) levels. However, analyzing social scenes involving FCGs is also highly challenging due to the difficulty in extracting behavioral cues such as target locations, their speaking activity and head/body pose due to crowdedness and presence of extreme occlusions. To this end, we propose SALSA, a novel dataset facilitating multimodal and Synergetic sociAL Scene Analysis, and make two main contributions to research on automated social interaction analysis: (1) SALSA records social interactions among 18 participants in a natural, indoor environment for over 60 minutes, under the poster presentation and cocktail party contexts presenting difficulties in the form of low-resolution images, lighting variations, numerous occlusions, reverberations and interfering sound sources; (2) To alleviate these problems we facilitate multimodal analysis by recording the social interplay using four static surveillance cameras and sociometric badges worn by each participant, comprising the microphone, accelerometer, bluetooth and infrared sensors. In addition to raw data, we also provide annotations concerning individuals' personality as well as their position, head, body orientation and F-formation information over the entire event duration. Through extensive experiments with state-of-the-art approaches, we show (a) the limitations of current methods and (b) how the recorded multiple cues synergetically aid automatic analysis of social interactions. SALSA is available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure

Effective Identity Management on Mobile Devices Using Multi-Sensor Measurements

Due to the dramatic increase in popularity of mobile devices in the past decade, sensitive user information is stored and accessed on these devices every day. Securing sensitive data stored and accessed from mobile devices, makes user-identity management a problem of paramount importance. The tension between security and usability renders the task of user-identity verification on mobile devices challenging. Meanwhile, an appropriate identity management approach is missing since most existing technologies for user-identity verification are either one-shot user verification or only work in restricted controlled environments. To solve the aforementioned problems, we investigated and sought approaches from the sensor data generated by human-mobile interactions. The data are collected from the on-board sensors, including voice data from microphone, acceleration data from accelerometer, angular acceleration data from gyroscope, magnetic force data from magnetometer, and multi-touch gesture input data from touchscreen. We studied the feasibility of extracting biometric and behaviour features from the on-board sensor data and how to efficiently employ the features extracted to perform user-identity verification on the smartphone device. Based on the experimental results of the single-sensor modalities, we further investigated how to integrate them with hardware such as fingerprint and Trust Zone to practically fulfill a usable identity management system for both local application and remote services control. User studies and on-device testing sessions were held for privacy and usability evaluation.Computer Science, Department o

Design Analytics Dashboards to Support Students and Instructors

Design coursework is iterative and continuously-evolving. Separation of digital tools used in design courses disaffects instructors’ and students’ iterative process experiences. As technology becomes increasingly integrated into design education, new opportunities arise for supporting the iterative, living process of design. These opportunities include providing on-demand, automatically computed insights to instructors, and facilitating instructor and student communication of feedback. I present a system that integrates support for design ideation with a learning analytics dashboard. The system enables instructors gain insights into a student's work across multiple dimensions. Instructors can view design work in the same environment in which students create it, which allows them to provide assessment and feedback in-context. I conducted semi-structured interviews, and recorded interaction logs over the course of an academic year to understand users' experiences. My research contributes to our understanding of how to present interactive, on-demand insights to instructors, as well as how to facilitate communication in an iterative process between instructors and students. Findings indicate benefits when systems enable instructors to contextualize creative work with assessment by integrating support for ideation with a learning analytics dashboard. Instructors are better able to track students and their work. Students are supported in reflecting on the relationship between assignments, and contextualizing instructor feedback with their work. We derive implications for contextualizing design with feedback to support creativity, learning, and teaching

Activity-driven content adaptation for effective video summarisation

In this paper, we present a novel method for content adaptation and video summarization fully implemented in compressed-domain. Firstly, summarization of generic videos is modeled as the process of extracted human objects under various activities/events. Accordingly, frames are classified into five categories via fuzzy decision including shot changes (cut and gradual transitions), motion activities (camera motion and object motion) and others by using two inter-frame measurements. Secondly, human objects are detected using Haar-like features. With the detected human objects and attained frame categories, activity levels for each frame are determined to adapt with video contents. Continuous frames belonging to same category are grouped to form one activity entry as content of interest (COI) which will convert the original video into a series of activities. An overall adjustable quota is used to control the size of generated summarization for efficient streaming purpose. Upon this quota, the frames selected for summarization are determined by evenly sampling the accumulated activity levels for content adaptation. Quantitative evaluations have proved the effectiveness and efficiency of our proposed approach, which provides a more flexible and general solution for this topic as domain-specific tasks such as accurate recognition of objects can be avoided

Employing Information and Communications Technologies in Homes and Cities for the Health and Well-Being of Older People

YesHe X and Sheriff RE (Eds.) Employing ICT in Homes and Cities for the Health and Well-Being of Older People. Workshop Proceedings of ICT4HOP’16. 15-17 Aug 2016. Sichuan University, Chengdu, China.British Council, Researcher Links, Newton Fund, NSF

Leveraging Douyin for Enhanced Learning Motivation: A Study on Educational Strategies and Student Attitudes

This study investigates the integration of Douyin, known globally as TikTok, in educational settings to enhance student learning motivation. The research aims to understand how the use of Douyin, a platform predominantly used for entertainment, can be effectively repurposed for educational purposes. The study is structured around several key areas: student engagement, self-regulation, teacher facilitation, peer collaboration, personalized learning, and student attitudes towards using Douyin in an educational context. A mixed-method approach is employed, involving both quantitative and qualitative data collection through surveys, interviews, and observational studies. The quantitative data assesses the impact of Douyin on various learning motivation factors, while qualitative data provides in-depth insights into student and teacher experiences. The findings suggest that when used strategically, Douyin can significantly enhance student engagement, promote active learning, and foster a positive learning environment. The study also highlights the importance of teachers’ roles in facilitating effective use of Douyin and the influence of student attitudes on learning outcomes.This research contributes to the growing body of knowledge on digital technology in education, offering practical implications for educators seeking to integrate social media platforms like Douyin into their teaching practices

Visual analysis of anatomy ontologies and related genomic information

Challenges in scientific research include the difficulty in obtaining overviews of the large amount of data required for analysis, and in resolving the differences in terminology used to store and interpret information in multiple, independently created data sets. Ontologies provide one solution for analysis involving multiple data sources, improving cross-referencing and data integration. This thesis looks at harnessing advanced human perception to reduce the cognitive load in the analysis of the multiple, complex data sets the bioinformatics user group studied use in research, taking advantage also of users’ domain knowledge, to build mental models of data that map to its underlying structure. Guided by a user-centred approach, prototypes were developed to provide a visual method for exploring users’ information requirements and to identify solutions for these requirements. 2D and 3D node-link graphs were built to visualise the hierarchically structured ontology data, to improve analysis of individual and comparison of multiple data sets, by providing overviews of the data, followed by techniques for detailed analysis of regions of interest. Iterative, heuristic and structured user evaluations were used to assess and refine the options developed for the presentation and analysis of the ontology data. The evaluation results confirmed the advantages that visualisation provides over text-based analysis, and also highlighted the advantages of each of 2D and 3D for visual data analysis.Overseas Research Students Awards SchemeJames Watt Scholarshi