Featuring, Detecting, and Visualizing Human Sentiment in Chinese Micro-Blog

2015-2016 > Academic research: refereed > Publication in refereed journa

EmoCo: Visual analysis of emotion coherence in presentation videos

Emotions play a key role in human communication and public presentations. Human emotions are usually expressed through multiple modalities. Therefore, exploring multimodal emotions and their coherence is of great value for understanding emotional expressions in presentations and improving presentation skills. However, manually watching and studying presentation videos is often tedious and time-consuming. There is a lack of tool support to help conduct an efficient and in-depth multi-level analysis. Thus, in this paper, we introduce EmoCo, an interactive visual analytics system to facilitate efficient analysis of emotion coherence across facial, text, and audio modalities in presentation videos. Our visualization system features a channel coherence view and a sentence clustering view that together enable users to obtain a quick overview of emotion coherence and its temporal evolution. In addition, a detail view and word view enable detailed exploration and comparison from the sentence level and word level, respectively. We thoroughly evaluate the proposed system and visualization techniques through two usage scenarios based on TED Talk videos and interviews with two domain experts. The results demonstrate the effectiveness of our system in gaining insights into emotion coherence in presentations.Comment: 11 pages, 8 figures. Accepted by IEEE VAST 201

Time sequencing the TRAP-18 indicators

A time sequence analysis is conducted on 125 lone-actor terrorists, most of whom mounted attacks in Europe and North America, utilizing the Terrorist Radicalization Assessment Protocol (TRAP-18), a structured professional judgment instrument with demonstrable interrater reliability and criterion, discriminant, and predictive validity. Both frequency filters (≥3) and coefficient filters (>.50) were applied to the data. Results indicate that virtually all distal characteristics, such as criminal violence, mental disorder, and ideology, preceded the proximal warning behaviors, such as pathway, fixation, identification, leakage, last resort, and directly communicated threats. Indicators that were “gatekeepers” and “turning point events” were also identified (Taylor et al., 2008). The time sequence analysis further validates the model of the TRAP-18 as a risk instrument for the assessment and management of lone-actor terrorist violence. (PsycInfo Database Record (c) 2021 APA, all rights reserved

Using Social Media Websites to Support Scenario-Based Design of Assistive Technology

Indiana University-Purdue University Indianapolis (IUPUI)Having representative users, who have the targeted disability, in accessibility studies is vital to the validity of research findings. Although it is a widely accepted tenet in the HCI community, many barriers and difficulties make it very resource-demanding for accessibility researchers to recruit representative users. As a result, researchers recruit non-representative users, who do not have the targeted disability, instead of representative users in accessibility studies. Although such an approach has been widely justified, evidence showed that findings derived from non-representative users could be biased and even misleading. To address this problem, researchers have come up with different solutions such as building pools of users to recruit from. But still, the data is not widely available and needs a lot of effort and resource to build and maintain. On the other hand, online social media websites have become popular in the last decade. Many online communities have emerged that allow online users to discuss health-related subjects, exchange useful information, or provide emotional support. A large amount of data accumulated in such online communities have gained attention from researchers in the healthcare domain. And many researches have been done based on data from social media websites to better understand health problems to improve the wellbeing of people. Despite the increasing popularity, the value of data from social media websites for accessibility research remains untapped. Hence, my work aims to create methods that could extract valuable information from data collected on social media websites for accessibility practitioners to support their design process. First, I investigate methods that enable researchers to effectively collect representative data from social media websites. More specifically, I look into machine learning approaches that could allow researchers to automatically identify online users who have disabilities (representative users). Second, I investigate methods that could extract useful information from user-generated free-text using techniques drawn from the information extraction domain. Last, I explore how such information should be visualized and presented for designers to support the scenario-based design process in accessibility studies

The role of visualisations in social media monitoring systems

Social-Media streams are constantly supplying vast volumes of real-time User Generated Content through platforms such as Twitter, Facebook, and Instagram, which makes it a challenge to monitor and understand. Understanding social conversations has now become a major interest for businesses, PR and advertising agencies, as well as law enforcement and government bodies. Monitoring of social-media allows us to observe large numbers of spontaneous, real-time interactions and varied expression of opinion, often fleeting and private. However, human, expert monitoring is generally unfeasible due to the high volumes of data. This has been a major reason for recent research and development work looking at automated social-media monitoring systems. Such systems often keep the human "out of the loop" as an NLP (Natural Language Processing) pipeline and other data-mining algorithms deal with analysing and extracting features and meaning from the data. This is plagued by a variety of problems, mostly due to the heterogenic, inconsistent and context-poor nature of social-media data, where as a result the accuracy and efficacy of such systems suffers. Nevertheless, automated social-media monitoring systems provide for a scalable, streamlined and often efficient way of dealing with big-data streams. The integration of processing outputs from automated systems and feedback to human experts is a challenge and deserves to be addressed in research literature. This paper will establish the role of the human in the social-media monitoring loop, based on prior systems work in this area. The focus of our investigation will be on use of visualisations for effective feedback to human experts. A specific, custom built system’s case-study in a social-media monitoring scenario will be considered and suggestions on how to bring back the human “into the loop” will be provided. Also some related ethical questions will be briefly considered. It is hoped that this work will inform and provide valuable insight to help improve development of automated social-media monitoring systems

Mining Social Media and Structured Data in Urban Environmental Management to Develop Smart Cities

This research presented the deployment of data mining on social media and structured data in urban studies. We analyzed urban relocation, air quality and traffic parameters on multicity data as early work. We applied the data mining techniques of association rules, clustering and classification on urban legislative history. Results showed that data mining could produce meaningful knowledge to support urban management. We treated ordinances (local laws) and the tweets about them as indicators to assess urban policy and public opinion. Hence, we conducted ordinance and tweet mining including sentiment analysis of tweets. This part of the study focused on NYC with a goal of assessing how well it heads towards a smart city. We built domain-specific knowledge bases according to widely accepted smart city characteristics, incorporating commonsense knowledge sources for ordinance-tweet mapping. We developed decision support tools on multiple platforms using the knowledge discovered to guide urban management. Our research is a concrete step in harnessing the power of data mining in urban studies to enhance smart city development

Bias-Aware Design for Informed Decisions: Raising Awareness of Self-Selection Bias in User Ratings and Reviews

People often take user ratings and reviews into consideration when shopping for products or services online. However, such user-generated data contains self-selection bias that could affect people decisions and it is hard to resolve this issue completely by algorithms. In this work, we propose to raise the awareness of the self-selection bias by making three types of information concerning user ratings and reviews transparent. We distill these three pieces of information (reviewers experience, the extremity of emotion, and reported aspects) from the definition of self-selection bias and exploration of related literature. We further conduct an online survey to assess the perceptions of the usefulness of such information and identify the exact facets people care about in their decision process. Then, we propose a visual design to make such details behind user reviews transparent and integrate the design into an experimental website for evaluation. The results of a between-subjects study demonstrate that our bias-aware design significantly increases the awareness of bias and their satisfaction with decision-making. We further offer a series of design implications for improving information transparency and awareness of bias in user-generated content