A multi-disciplinary co-design approach to social media sensemaking with text mining

This thesis presents the development of a bespoke social media analytics platform called Sentinel using an event driven co-design approach. The performance and outputs of this system, along with its integration into the routine research methodology of its users, were used to evaluate how the application of an event driven co-design approach to system design improves the degree to which Social Web data can be converted into actionable intelligence, with respect to robustness, agility, and usability. The thesis includes a systematic review into the state-of-the-art technology that can support real-time text analysis of social media data, used to position the text analysis elements of the Sentinel Pipeline. This is followed by research chapters that focus on combinations of robustness, agility, and usability as themes, covering the iterative developments of the system through the event driven co-design lifecycle. Robustness and agility are covered during initial infrastructure design and early prototyping of bottom-up and top-down semantic enrichment. Robustness and usability are then considered during the development of the Semantic Search component of the Sentinel Platform, which exploits the semantic enrichment developed in the prototype, alpha, and beta systems. Finally, agility and usability are used whilst building upon the Semantic Search functionality to produce a data download functionality for rapidly collecting corpora for further qualitative research. These iterations are evaluated using a number of case studies that were undertaken in conjunction with a wider research programme, within the field of crime and security, that the Sentinel platform was designed to support. The findings from these case studies are used in the co-design process to inform how developments should evolve. As part of this research programme the Sentinel platform has supported the production of a number of research papers authored by stakeholders, highlighting the impact the system has had in the field of crime and security researc

Enhancing Free-text Interactions in a Communication Skills Learning Environment

Learning environments frequently use gamification to enhance user interactions.Virtual characters with whom players engage in simulated conversations often employ prescripted dialogues; however, free user inputs enable deeper immersion and higher-order cognition. In our learning environment, experts developed a scripted scenario as a sequence of potential actions, and we explore possibilities for enhancing interactions by enabling users to type free inputs that are matched to the pre-scripted statements using Natural Language Processing techniques. In this paper, we introduce a clustering mechanism that provides recommendations for fine-tuning the pre-scripted answers in order to better match user inputs

The student-produced electronic portfolio in craft education

The authors studied primary school students’ experiences of using an electronic portfolio in their craft education over four years. A stimulated recall interview was applied to collect user experiences and qualitative content analysis to analyse the collected data. The results indicate that the electronic portfolio was experienced as a multipurpose tool to support learning. It makes the learning process visible and in that way helps focus on and improves the quality of learning. © ISLS.Peer reviewe

Cognitive architecture of multimodal multidimensional dialogue management

Numerous studies show that participants of real-life dialogues happen to get involved in rather dynamic non-sequential interactions. This challenges the dialogue system designs based on a reactive interlocutor paradigm and calls for dialog systems that can be characterised as a proactive learner, accomplished multitasking planner and adaptive decision maker. Addressing this call, the thesis brings innovative integration of cognitive models into the human-computer dialogue systems. This work utilises recent advances in Instance-Based Learning of Theory of Mind skills and the established Cognitive Task Analysis and ACT-R models. Cognitive Task Agents, producing detailed simulation of human learning, prediction, adaption and decision making, are integrated in the multi-agent Dialogue Man-ager. The manager operates on the multidimensional information state enriched with representations based on domain- and modality-specific semantics and performs context-driven dialogue acts interpretation and generation. The flexible technical framework for modular distributed dialogue system integration is designed and tested. The implemented multitasking Interactive Cognitive Tutor is evaluated as showing human-like proactive and adaptive behaviour in setting goals, choosing appropriate strategies and monitoring processes across contexts, and encouraging the user exhibit similar metacognitive competences

Improving Hybrid Brainstorming Outcomes with Scripting and Group Awareness Support

Previous research has shown that hybrid brainstorming, which combines individual and group methods, generates more ideas than either approach alone. However, the quality of these ideas remains similar across different methods. This study, guided by the dual-pathway to creativity model, tested two computer-supported scaffolds – scripting and group awareness support – for enhancing idea quality in hybrid brainstorming. 94 higher education students,grouped into triads, were tasked with generating ideas in three conditions. The Control condition used standard hybrid brainstorming without extra support. In the Experimental 1 condition, students received scripting support during individual brainstorming, and students in the Experimental 2 condition were provided with group awareness support during the group phase in addition. While the quantity of ideas was similar across all conditions, the Experimental 2 condition produced ideas of higher quality, and the Experimental 1 condition also showed improved idea quality in the individual phase compared to the Control condition

CLINICAL AND SOCIAL PATHWAYS TO CARE: A COMPUTATIONAL EXAMINATION OF SOCIAL MEDIA FOR MENTAL HEALTH CARE

In the last decade, powered by connectivity to large social networks and advances in collecting and analyzing digital traces of individuals from social media platforms, researchers have gleaned rich insights into individuals’ and populations’ mental health states and experiences, including their moods, emotions, social interactions, language, and communication patterns. Using these inferences, researchers have been able to study support-seeking behaviors, distinguishing patterns, risk markers, and diagnosis states for mental illnesses from social media data, promising a fundamental change in mental health care. What we need next in this line of work is for data and algorithms based on social media to be contextualized in people’s pathways to mental health care. However, there are several challenges and unanswered questions that present hurdles. First, gaps exist in the psychometric validity of social media based measurements of behaviors and the utility of these inferences in predicting clinical outcomes in patient populations. Second, if social media can act as an intervention platform, outside of discrete events, a holistic understanding of its role in people’s lives along the course of a mental illness is crucial. Lastly, several questions remain around the ethical implications of research practices in engaging with a vulnerable population subject to this research. This thesis charts out empirical and critical understandings and develops novel computational techniques to ethically and holistically examine how social media can be employed to support mental health care. Focusing on schizophrenia, one of the most debilitating and stigmatizing of mental illnesses, this thesis contributes a deeper understanding on pathways to care via social media along three themes: 1) prediction of clinical mental health states from social media data to support clinical interventions, 2) understanding online self-disclosure and social support as pathways to social care, and 3) the intersection of social and clinical pathways to care along the course of mental illness. In doing so, this work combines theories from social psychology, computer-mediated communication, and clinical literature with machine learning, statistical modeling, and natural language analysis methods applied on large-scale behavioral data from social media platforms. Together, this work contributes novel methodologies and human-centered algorithmic design frameworks to understand the efficacy of social media as a mental health intervention platform, informing clinicians, researchers, and designers who engage in developing and deploying interventions for mental health and well-being.Ph.D

Computational and Causal Approaches on Social Media and Multimodal Sensing Data: Examining Wellbeing in Situated Contexts

A core aspect of our lives is often embedded in the communities we are situated in. The interconnectedness of our interactions and experiences intertwines our situated context with our wellbeing. A better understanding of wellbeing will help us devise proactive and tailored support strategies. However, existing methodologies to assess wellbeing suffer from limitations of scale and timeliness. These limitations are surmountable by social and ubiquitous technologies. Given its ubiquity and wide use, social media can be considered a “passive sensor” that can act as a complementary source of unobtrusive, real-time, and naturalistic data to infer wellbeing. This dissertation leverages social media in concert with multimodal sensing data, which facilitate analyzing dense and longitudinal behavior at scale. This work adopts machine learning, natural language, and causal inference analysis to infer wellbeing of individuals and collectives, particularly in situated communities, such as college campuses and workplaces. Before incorporating sensing modalities in practice, we need to account for confounds. One such confound that might impact behavior change is the phenomenon of “observer effect” --- that individuals may deviate from their typical or otherwise normal behavior because of the awareness of being “monitored”. I study this problem by leveraging the potential of longitudinal and historical behavioral data through social media. Focused on a multimodal sensing study, I conduct a causal study to measure observer effect in social media behavior, and explain the observations through existing theory in psychology and social science. The findings provide recommendations to correcting biases due to observer effect in social media sensing for human behavior and wellbeing. The novelties and contributions of this dissertation are four-fold. First, I use social media data that uniquely captures the behavior of situated communities. Second, I adopt theory-driven computational and causal methods to make conclusive research claims on wellbeing dynamics. Third, I address major challenges with methods to combine social media with multimodal sensing data for a comprehensive understanding of human behavior. Fourth, I draw interpretations and explanations of online-data-driven offline inferences. This dissertation situates the findings in an interdisciplinary context, including psychology and social science, and bears implications from theoretical, practical, design, methodological, and ethical perspectives catering to various stakeholders, including researchers, practitioners, and policymakers.Ph.D