SpeechMirror: A Multimodal Visual Analytics System for Personalized Reflection of Online Public Speaking Effectiveness

As communications are increasingly taking place virtually, the ability to present well online is becoming an indispensable skill. Online speakers are facing unique challenges in engaging with remote audiences. However, there has been a lack of evidence-based analytical systems for people to comprehensively evaluate online speeches and further discover possibilities for improvement. This paper introduces SpeechMirror, a visual analytics system facilitating reflection on a speech based on insights from a collection of online speeches. The system estimates the impact of different speech techniques on effectiveness and applies them to a speech to give users awareness of the performance of speech techniques. A similarity recommendation approach based on speech factors or script content supports guided exploration to expand knowledge of presentation evidence and accelerate the discovery of speech delivery possibilities. SpeechMirror provides intuitive visualizations and interactions for users to understand speech factors. Among them, SpeechTwin, a novel multimodal visual summary of speech, supports rapid understanding of critical speech factors and comparison of different speech samples, and SpeechPlayer augments the speech video by integrating visualization of the speaker's body language with interaction, for focused analysis. The system utilizes visualizations suited to the distinct nature of different speech factors for user comprehension. The proposed system and visualization techniques were evaluated with domain experts and amateurs, demonstrating usability for users with low visualization literacy and its efficacy in assisting users to develop insights for potential improvement.Comment: Main paper (11 pages, 6 figures) and Supplemental document (11 pages, 11 figures). Accepted by VIS 202

Online annotations tools for micro-level human behavior labeling on videos

Abstract. Successful machine learning and computer vision approach generally require significant amounts of annotated data for learning. These methods including identification, retrieval, classification of events, and analysis of human behavior from a video. Micro-level human behavior analysis usually requires laborious efforts for obtaining the precise labels. As the quantity of online video grows, the crowdsourcing approach provides a method for workers without a professional background to complete the annotation task. These workers require training to understand implicit knowledge of human behavior. The motivation of this study was to enhance the interaction between annotation workers for training purposes. By observing experienced local researchers in Oulu, the key problem with annotation is the precision of the results. The goal of this study was to provide training tools for people to improve the label quality, it illustrates the importance of training. In this study, a new annotation tool was developed to test workers’ performance in reviewing other annotations. This tool filters very noisy input by comment and vote feature. The result indicated that users were more likely to annotate micro behavior and time that refer to other opinions, and it was a more effective and reliable way to train. Besides, this study reported the development process with React and Firebase, it emphasized the use of more Web resources and tools to develop annotation tools

Searching, navigating, and recommending movies through emotions: A scoping review

Movies offer viewers a broad range of emotional experiences, providing entertainment, and meaning. Following the PRISMA-ScR guidelines, we reviewed the literature on digital systems designed to help users search and browse movie libraries and offer recommendations based on emotional content. Our search yielded 83 eligible documents (published between 2000 and 2021). We identified 22 case studies, 34 empirical studies, 26 proof of concept, and one theoretical paper. User transactions (e.g., ratings, tags) were the preferred source of information. The documents examined approached emotions from both categorical (n=35) and dimensional (n=18) perspectives, and nine documents offer a combination of both approaches. Although there are several authors mentioned, the references used are frequently dated, and 12 documents do not mention the author or the model used. We identified 61 words related to emotion or affect. Documents presented on average 1.36 positive terms and 2.64 negative terms. Sentiment analysis () is frequently used for emotion identification, followed by subjective evaluations (n= 15), movie low-level audio and visual features (n = 11), and face recognition technologies (n = 8). We discuss limitations and offer a brief review of current emotion models and research.info:eu-repo/semantics/publishedVersio

Authentication of Students and Students’ Work in E-Learning : Report for the Development Bid of Academic Year 2010/11

Global e-learning market is projected to reach $107.3 billion by 2015 according to a new report by The Global Industry Analyst (Analyst 2010). The popularity and growth of the online programmes within the School of Computer Science obviously is in line with this projection. However, also on the rise are students’ dishonesty and cheating in the open and virtual environment of e-learning courses (Shepherd 2008). Institutions offering e-learning programmes are facing the challenges of deterring and detecting these misbehaviours by introducing security mechanisms to the current e-learning platforms. In particular, authenticating that a registered student indeed takes an online assessment, e.g., an exam or a coursework, is essential for the institutions to give the credit to the correct candidate. Authenticating a student is to ensure that a student is indeed who he says he is. Authenticating a student’s work goes one step further to ensure that an authenticated student indeed does the submitted work himself. This report is to investigate and compare current possible techniques and solutions for authenticating distance learning student and/or their work remotely for the elearning programmes. The report also aims to recommend some solutions that fit with UH StudyNet platform.Submitted Versio

A System to Generate SignWriting for Video Tracks Enhancing Accessibility of Deaf People

Video content has increased much on the Internet during last years. In spite of the efforts of different organizations and governments to increase the accessibility of websites, most multimedia content on the Internet is not accessible. This paper describes a system that contributes to make multimedia content more accessible on the Web, by automatically translating subtitles in oral language to SignWriting, a way of writing Sign Language. This system extends the functionality of a general web platform that can provide accessible web content for different needs. This platform has a core component that automatically converts any web page to a web page compliant with level AA of WAI guidelines. Around this core component, different adapters complete the conversion according to the needs of specific users. One adapter is the Deaf People Accessibility Adapter, which provides accessible web content for the Deaf, based on SignWritting. Functionality of this adapter has been extended with the video subtitle translator system. A first prototype of this system has been tested through different methods including usability and accessibility tests and results show that this tool can enhance the accessibility of video content available on the Web for Deaf people

EMOTION BASED MUSIC PLAYER

The work presents described the development of Emotion Based Music Player, which is a computer application meant for all type of users, specifically the music lovers. Due to the troublesome workloads in songs selection, most people will choose to randomly play the songs in the playlist. As a result, some of the songs selected not matching the users’ current emotion. Moreover, there is no commonly used music player which able to play the songs based on user’s emotion. The proposed model is able to extract user’s facial expression and thus detect user’s emotion. The music player in the proposed model will then play the songs according to the category of emotion detected. It is aimed to provide a better enjoyment to music lovers in music listening. The scope of emotions in the proposed model involve normal, sad, surprise and happy. The system involves the major of image processing and facial detection technologies. The input for this proposed model is the .jpeg format still images which available online. The performance of this model is evaluated by loading forty still images (ten for each emotion category) into the proposed model to test on the accuracy in detecting the emotions. Based on the testing result, the proposed model has the Recognition Rate of 85%

MIXTURE FEATURE EXTRACTION BASED ON LOCAL BINARY PATTERN AND GREY-LEVEL CO-OCCURRENCE MATRIX TECHNIQUES FOR MOUTH EXPRESSION RECOGNITION

Some academics struggle to recognize facial emotions based on pattern recognition. In general, this recognition utilizes all facial features. However, this study was limited to identifying facial emotions in a single facial region. In this study, lips, one of the facial features that can reveal a person's expression, are utilized. Using a combination of local binary pattern feature extraction (LBP) and grey level co-occurrence matrix (GLCM) methods and a multiclass support vector machine classification approach for feature extraction in facial images. The concept begins with image segmentation to create an image of a mouth. Experiments were also conducted for various tests, and the outcomes of these experiments revealed a recognition performance of up to 95%. This result was obtained through experiments in which 10% to 40% of the data were evaluated. These findings are beneficial and can be applied to expression recognition in online learning media to monitor the audience's condition directly