Detecting and Modelling Stress Levels in E-Learning Environment Users

A modern Intelligent Tutoring System (ITS) should be sentient of a learner's cognitive and affective states, as a learner’s performance could be affected by motivational and emotional factors. It is important to design a method that supports low-cost, task-independent and unobtrusive sensing of a learner’s cognitive and affective states, to improve a learner's experience in e-learning, as well as to enable personalized learning. Although tremendous related affective computing research were done in this area, there is a lack of empirical research that can automatically measure a learner's stress using objective methods. This research is set to examine how an objective stress measurement model can be developed, to compute a learner’s cognitive and emotional stress automatically using mouse and keystroke dynamics. To ensure the measurement is not affected even if the user switches between tasks, three preliminary research experiments were carried out based on three common tasks during e-learning − search, assessment and typing. A stress measurement model was then built using the datasets collected from the experiments. Three stress classifiers were tested, namely certainty factors, feedforward back-propagation neural network and adaptive neuro-fuzzy inference system. The best classifier was then integrated into the ITS stress inference engine, which is designed to decide necessary adaptation, and to provide analytical information of learners' performances, which include stress levels and learners’ behaviours when answering questions

Examining the sense of agency in human-computer interaction

Humans are agents, we feel that we control the course of events on our everyday life. This refers to the Sense of Agency (SoA). This experience is not only crucial in our daily life, but also in our interaction with technology. When we manipulate a user interface (e.g., computer, smartphone, etc.), we expect that the system responds to our input commands with feedback, as we desire to feel that we are in charge of the interaction. If this interplay elicits a SoA, then the user will perceive an instinctive feeling of “I am controlling this”. Although research in Human-Computer Interaction (HCI) pursuits the design of intuitive and responsive systems, most of the current studies have been focussed mainly on interaction techniques (e.g., software-hardware) and User Experience (UX) (e.g., comfort, usability, etc.), and very little has been investigated in terms of the SoA i.e., the conscious experience of being in control regarding the interaction. In this thesis, we present an experimental exploration of the role of the SoA in interaction paradigms typical of HCI. After two chapters of introduction and related work, we describe a series of studies that explore agency implication in interaction with systems through human senses such as vision, audio, touch and smell. Chapter 3 explores the SoA in mid-air haptic interaction through touchless actions. Then, Chapter 4 examines agency modulation through smell and its application for olfactory interfaces. Chapter 5 describes two novel timing techniques based on auditory and haptic cues that provide alternative timing methods to the traditional Libet clock. Finally, we conclude with a discussion chapter that highlights the importance of our SoA during interactions with technology as well as the implications of the results found, in the design of user interfaces

Advances in Human-Robot Interaction

Rapid advances in the field of robotics have made it possible to use robots not just in industrial automation but also in entertainment, rehabilitation, and home service. Since robots will likely affect many aspects of human existence, fundamental questions of human-robot interaction must be formulated and, if at all possible, resolved. Some of these questions are addressed in this collection of papers by leading HRI researchers

ANICE : An Artificial Neuro-Linguistic Interactive Computer Entity

Mental health problems are hard to talk about, especially when the questions asked do not allow the individual to answer freely. That is the case for most inquiries, where questions usually request very restricted answers like yes or no. This thesis proposes a chatbot that tries to avoid the problem of restricting users to small answers. The chatbot will focus on people feeling burned out due to stress related to their studies. The chatbot tries to replicate two forms with questions about burnout that are used as guidelines. Both these forms are developed based on questions done by psychologists. Because rule-based chatbots have a limited vocabulary, natural language understand- ing and neural-based techniques are tested and validated to see if the chatbot performs well using these techniques. The techniques tested are word2vec and spacy components. The evaluation results show that it is feasible to implement a chatbot that uses rules and also techniques for natural language processing. Additionally, the tests did indicate that both spacy and word2vec are great resources for NLU. Word2vec proves to perform slightly better at specific times related to identifying intents that are domain-specific. Finally, the results from the users experience show that this is a promising work that could help students dealing with burnout.Problemas de saúde mental são um tema difícil de abordar, especialmente quando as perguntas feitas não permitem ao indivíduo responder livremente. Este é o caso da maio- ria dos inquéritos, onde as perguntas geralmente exigem respostas muito restritas, como sim ou não. Esta tese propõe um chatbot que tenta evitar o problema de restringir os utilizadores a pequenas respostas. O chatbot concentrar-se-á em utilizadores que se sen- tem esgotados devido ao stress relacionado com os estudos. O chatbot tenta replicar dois formulários com perguntas sobre burnout, isto é, estes formulários são utilizados como diretrizes. Ambos os formulários são desenvolvidos com base em perguntas feitas por psicólogos. Como os chatbots baseados em regras têm um vocabulário limitado, a compreensão da linguagem natural e as técnicas baseadas em redes neuronais são testadas e validadas para ver se o chatbot tem um bom funcionamento utilizando essas técnicas. As técnicas baseadas em redes neuronais que são testadas são o word2vec e componentes spacy. Os resultados da avaliação mostram que é viável implementar um chatbot que uti- lize regras e também técnicas de processamento de linguagem natural. Além disso, os testes indicam que tanto os componentes spacy quanto o word2vec são ótimos recursos para processamento de linguagem natural. O Word2vec tem um desempenho um pouco melhor em momentos específicos relacionados à identificação de intenções do domínio de estudo. Por fim, os resultados da experiência dos utilizadores mostram que este é um trabalho promissor que pode ajudar os utilizadores a lidar com o burnout

Advanced Augmentative and Alternative Communication System Based in Physiological Control

Dyskinetic Cerebral Palsy (DCP) is mainly characterized by alterations in muscle tone and involuntary movements. Therefore, these people present with difficulties in coordination and movement control, which makes walking difficult and affects their posture when seated. Additionally, their cognitive performance varies between being completely normal and severe mental retardation. People with DCP were selected as the objective of this thesis due to their multiple and complex limitations (speech problems and motor control) and because their capabilities have a great margin for improvement thanks to physiological control systems. Given their communication difficulties, some people with DCP have good motor con-trol and can communicate with written language. However, most have difficulty using Augmentative and Alternative Communication (AAC) systems. People with DCP gen-erally use concept boards to indicate the idea they want to communicate. However, most communication solutions available today are based on proprietary software that makes it difficult to customize the concept board and this type of control system. This is the motivation behind this thesis, with the aim of creating an interface with characteristics, able to be adapted to the user needs and limitations. Thus, this thesis proposes an Augmentative and Alternative Communication System for people with DCP based on physiological control. In addition, an innovative system for direct con-trol of concept boards with EMG is proposed. This control system is based on a physi-cal model that reproduces the muscular mechanical response (stiffness, inertia and viscosity). It allows for a selection of elements thanks to small pulses of EMG signal with sensors on a muscle with motor control. Its main advantage is the possibility of correcting errors during selection associated with uncontrolled muscle impulses, avoid-ing sustained muscle effort and thus reduced fatigue.La Parálisis Cerebral de tipo Discinésica (DCP) se caracteriza principalmente por las alteraciones del tono muscular y los movimientos involuntarios. Por ello, estos pacientes presentan dificultades en la coordinación y en el control de movimientos, lo cual les dificulta el caminar y afecta su postura cuando están sentados. Cabe resaltar que la capacidad cognitiva de las personas con DCP puede variar desde completamente normal, hasta un retraso mental severo. Las personas con DCP han sido seleccionadas como objetivo de esta tesis ya el margen de mejora de sus capacidades es amplio gracias a sistemas de control fisiológico, debido a sus múltiples y complejas limitaciones (problemas de habla y control motor). Debido a sus dificultades de comunicación, algunas personas con DCP se pueden comunicar con lenguaje escrito, siempre y cuando tenga un buen control motor. Sin embargo, la mayoría tienen dificultades para usar sistemas de Comunicación Aumentativos y Alternativos (AAC). De hecho, las personas con DCP utilizan generalmente tableros de conceptos para indicar la idea que quieren transmitir. Sin embargo, la mayoría las soluciones de comunicación disponibles en la actualidad están basadas en software propietario que hacen difícil la personalización del tablero de conceptos y el tipo de sistema de control. Es aquí donde surge esta tesis, con el objetivo de crear una interfaz con esas características, capaz de adaptarse a las necesidades y limitaciones del usuario. De esta forma, esta tesis propone un sistema de comunicación aumentativo y alternativo para personas con DCP basado en control fisiológico. Además, se propone un Sistema innovador de control directo sobre tableros de conceptos basado en EMG. Este Sistema de control se basa en un modelo físico que reproduce la respuesta mecánica muscular (basado en parámetros como Rigidez, Inercia y Viscosidad), permitiendo la selección de elementos gracias a pequeños pulsos de señal EMG con sensores sobre un músculo con control motor. Sus principales ventajas son la posibilidad de corregir errores durante la selección asociado a los impulsos musculares no controlados, evitar el esfuerzo muscular mantenido para alcanzar un nivel y reducir la fatiga.La Paràlisi Cerebral de tipus Discinèsica (DCP) es caracteritza principalment per les alteracions del to muscular i els moviments involuntaris. Per açò, aquests pacients presenten dificultats en la coordinació i en el control de moviments, la qual cosa els dificulta el caminar i afecta la seua postura quan estan asseguts. Cal ressaltar que la capacitat cognitiva de les persones amb DCP pot variar des de completament normal, fins a un retard mental sever. Les persones amb DCP han sigut seleccionades com a objectiu d'aquesta tesi ja el marge de millora de les seues capacitats és ampli gràcies a sistemes de control fisiològic, a causa dels seus múltiples i complexes limitacions (problemes de parla i control motor). A causa de les seues dificultats de comunicació, algunes persones amb DCP es poden comunicar amb llenguatge escrit, sempre que tinga un bon control motor. No obstant açò, la majoria tenen dificultats per a usar sistemes de Comunicació Augmentatius i Alternatius (AAC). De fet, les persones amb DCP utilitzen generalment taulers de conceptes per a indicar la idea que volen transmetre. No obstant açò, la majoria les solucions de comunicació disponibles en l'actualitat estan basades en programari propietari que fan difícil la personalització del tauler de conceptes i el tipus de sistema de control. És ací on sorgeix aquesta tesi, amb l'objectiu de crear una interfície amb aqueixes característiques, capaç d'adaptar-se a les necessitats i limitacions de l'usuari. D'aquesta forma, aquesta tesi proposa un sistema de comunicació augmentatiu i alternatiu per a persones amb DCP basat en control fisiològic. A més, es proposa un sistema innovador de control directe sobre taulers de conceptes basat en EMG. Aquest sistema de control es basa en un model físic que reprodueix la resposta mecànica muscular (basat en paràmetres com a Rigidesa, Inèrcia i Viscositat), permetent la selecció d'elements gràcies a xicotets polsos de senyal EMG amb sensors sobre un múscul amb control motor. Els seus principals avantatges són la possibilitat de corregir errors durant la selecció associat als impulsos musculars no controlats, evitar l'esforç muscular mantingut per a aconseguir un nivell i reduir la fatiga.Díaz Pineda, JA. (2017). Advanced Augmentative and Alternative Communication System Based in Physiological Control [Tesis doctoral no publicada]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/90418TESI

Body gestures recognition for human robot interaction

In this project, a solution for human gesture classification is proposed. The solution uses a Deep Learning model and is meant to be useful for non-verbal communication between humans and robots. The State-of-the-Art is researched in an effort to achieve a model ready to work with natural gestures without restrictions. The research will focus on the creation of a temPoral bOdy geSTUre REcognition model (POSTURE) that can recognise continuous gestures performed in real-life situations. The suggested model takes into account spatial and temporal components so as to achieve the recognition of more natural and intuitive gestures. In a first step, a framework extracts from all the images the corresponding landmarks for each of the body joints. Next, some data filtering techniques are applied with the aim of avoiding problems related with the data. Afterwards, the filtered data is input into an State-of-the-Art Neural Network. And finally, different neural network configurations and approaches are tested to find the optimal performance. The obtained outcome shows the research has been done in the right track and how, despite of the dataset problems found, even better results can be achievedObjectius de Desenvolupament Sostenible::9 - Indústria, Innovació i Infraestructur

Emotional recognition in computing

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University 8/4/2010.Emotions are fundamental to human lives and decision-making. Understanding and expression of emotional feeling between people forms an intricate web. This complex interactional phenomena, is a hot topic for research, as new techniques such as brain imaging give us insights about how emotions are tied to human functions. Communication of emotions is mixed with communication of other types of information (such as factual details) and emotions can be consciously or unconsciously displayed. Affective computer systems, using sensors for emotion recognition and able to make emotive responses are under development. The increased potential for emotional interaction with products and services, in many domains, is generating much interest. Emotionally enhanced systems have potential to improve human computer interaction and so to improve how systems are used and what they can deliver. They may also have adverse implications such as creating systems capable of emotional manipulation of users. Affective systems are in their infancy and lack human complexity and capability. This makes it difficult to assess whether human interaction with such systems will actually prove beneficial or desirable to users. By using experimental design, a Wizard of Oz methodology and a game that appeared to respond to the user’s emotional signals with human-like capability, I tested user experience and reactions to a system that appeared affective. To assess users’ behaviour, I developed a novel affective behaviour coding system called ‘affectemes’. I found significant gains in user satisfaction and performance when using an affective system. Those believing the system responded to emotional signals blinked more frequently. If the machine failed to respond to their emotional signals, they increased their efforts to convey emotion, which might be an attempt to ‘repair’ the interaction. This work highlights how very complex and difficult it is to design and evaluate affective systems. I identify many issues for future work, including the unconscious nature of emotions and how they are recognised and displayed with affective systems; issues about the power of emotionally interactive systems and their evaluation; and critical ethical issues. These are important considerations for future design of systems that use emotion recognition in computing.EPSRC project grant (R81374/01

Socio-Cognitive and Affective Computing

Social cognition focuses on how people process, store, and apply information about other people and social situations. It focuses on the role that cognitive processes play in social interactions. On the other hand, the term cognitive computing is generally used to refer to new hardware and/or software that mimics the functioning of the human brain and helps to improve human decision-making. In this sense, it is a type of computing with the goal of discovering more accurate models of how the human brain/mind senses, reasons, and responds to stimuli. Socio-Cognitive Computing should be understood as a set of theoretical interdisciplinary frameworks, methodologies, methods and hardware/software tools to model how the human brain mediates social interactions. In addition, Affective Computing is the study and development of systems and devices that can recognize, interpret, process, and simulate human affects, a fundamental aspect of socio-cognitive neuroscience. It is an interdisciplinary field spanning computer science, electrical engineering, psychology, and cognitive science. Physiological Computing is a category of technology in which electrophysiological data recorded directly from human activity are used to interface with a computing device. This technology becomes even more relevant when computing can be integrated pervasively in everyday life environments. Thus, Socio-Cognitive and Affective Computing systems should be able to adapt their behavior according to the Physiological Computing paradigm. This book integrates proposals from researchers who use signals from the brain and/or body to infer people's intentions and psychological state in smart computing systems. The design of this kind of systems combines knowledge and methods of ubiquitous and pervasive computing, as well as physiological data measurement and processing, with those of socio-cognitive and affective computing