Neural Conversation Generation with Auxiliary Emotional Supervised Models

An important aspect of developing dialogue agents involves endowing a conversation system with emotion perception and interaction. Most existing emotion dialogue models lack the adaptability and extensibility of different scenes because of their limitation to require a specified emotion category or their reliance on a fixed emotional dictionary. To overcome these limitations, we propose a neural conversation generation with auxiliary emotional supervised model (nCG-ESM) comprising a sequence-to-sequence (Seq2Seq) generation model and an emotional classifier used as an auxiliary model. The emotional classifier was trained to predict the emotion distributions of the dialogues, which were then used as emotion supervised signals to guide the generation model to generate diverse emotional responses. The proposed nCG-ESM is flexible enough to generate responses with emotional diversity, including specified or unspecified emotions, which can be adapted and extended to different scenarios. We conducted extensive experiments on the popular dataset of Weibo post--response pairs. Experimental results showed that the proposed model was capable of producing more diverse, appropriate, and emotionally rich responses, yielding substantial gains in diversity scores and human evaluations.Peer reviewe

A Review on Human-Computer Interaction and Intelligent Robots

In the field of artificial intelligence, human–computer interaction (HCI) technology and its related intelligent robot technologies are essential and interesting contents of research. From the perspective of software algorithm and hardware system, these above-mentioned technologies study and try to build a natural HCI environment. The purpose of this research is to provide an overview of HCI and intelligent robots. This research highlights the existing technologies of listening, speaking, reading, writing, and other senses, which are widely used in human interaction. Based on these same technologies, this research introduces some intelligent robot systems and platforms. This paper also forecasts some vital challenges of researching HCI and intelligent robots. The authors hope that this work will help researchers in the field to acquire the necessary information and technologies to further conduct more advanced research

Criação de uma Framework para Melhorar a Experiência de Jogo através de Biofeedback

Na área da interação pessoa-máquina (IPM), tem havido recentemente, um forte interesse em novas formas de interação que complementam ou substituem a interação tradicional - o teclado e o rato. Estas novas formas de interação, como o biofeedback com a utilização de sinais fisiológicos, permitem aos jogadores uma experiência mais imersiva.Embora já investigada a utilização de mecanismos de biofeedback no contexto de videojogos, e principalmente, os mecanismos de biofeedback directo onde jogadores executam acções no jogo de forma explícita, não foram estudadas as limitações que os mecanismos de biofeedback têm quando aplicados a videojogos já existentes.O objetivo desta dissertação é o de estudar os efeitos dos mecanismos de biofeedback aplicados a videojogos comerciais pré-existentes - isto é, explorar a variedade de mecanismos que podem ser usados com uma mesma framework e perceber como o mesmo sensor pode servir a diferentes mecânicas. Também se pretende identificar as limitações de desenho e adaptação destes às mecânicas de jogo atuais. Para tal, foi desenvolvida uma framework capaz de recolher e interpretar os dados provenientes de diversos sensores fisiológicos e fornecer uma interface gráfica simples para a configuração de regras de ativação de mecânicas de jogo.A partir da integração desta framework no videojogo Skyrim, foi possível realizar um caso de estudo com 26 participantes, onde foram comparados dois cenários, um recorrendo à interação tradicional e outro aos mecanismos de interação fisiológica. Assim, foi possível avaliar o impacto destes mecanismos de interação através de questionários que permitiram avaliar a experiência de jogo dos participantes em ambos os cenários, consoante os graus de Divertimento, Facilidade de Uso, Originalidade, entre outras.Os resultados da dissertação permitem concluir que a versão biofeedback mostrou ser mais divertida, embora as mecânicas de jogo na versão original sejam de mais fácil utilização. A maioria dos participantes deu preferência ao videojogo com biofeedback. Este trabalho realça as vantagens que a aplicação de mecânicas de jogo aliadas à utilização do biofeedback podem trazer aos videojogos.In the field of Human-Computer Interaction (HCI), there has been recently a strong interest in new forms of interaction that complement or replace traditional - keyboard and mouse. These new forms of interaction, such as biofeedback with the use of physiological signals, allow players a more immersive experience.Although already investigated the use of biofeedback mechanisms in the context of videogames, and especially the mechanisms of direct biofeedback where players perform actions in the game explicitly, it hasn't been studied the limitations that biofeedback mechanisms have when applied to already existing videogames.The objective of this dissertation is to study the effects of biofeedback mechanisms applied to preexisting commercial videogames - that is, to explore the variety of mechanisms that can be used with the same framework and to understand how the same sensor can serve different mechanics. It is also intended to identify the design and adaptation limitations of this to current games' mechanics. To accomplish this, a framework was developed capable of collecting and interpreting data from several physiological sensors, and provide a simple graphical interface for the configuration of activation rules of the game mechanics. Through the integration of this framework in the videogame Skyrim, it was possible to carry out a study case with 26 participants, where we compared two scenarios, one using traditional interaction and the other physiological interaction. Thus, it was possible to evaluate the impact of these mechanisms of interaction through questionnaires that allowed the evaluation of participants' experience in both scenarios, depending on the degrees of Fun, Usability, Originality, among others.The results of the dissertation allow us to conclude that the biofeedback version proved to be more fun, although the game mechanics in the original version are more user-friendly. The majority of participants gave preference to the videogame with biofeedback. This work highlights the advantages that the application of game mechanics allied to the use of biofeedback can bring to videogames

The gesture's narrative : contemporary music for percussion

Musical performance gestures are recognized by the majority of theoreticians as a critical factor of a musical performance. The aim of the musical performance may consist in not only communicating the musical signs that form a musical piece, but conveying the meaningful succession of gestures, facial expressions and body movements. This meaningful succession, or otherwise the “gesture´s narrative” is assumed to be quite important for the process of directing the audience towards the intended interpretation. Recording music allowed audiences to listen to music without having to go to a musical event for this purpose. On the one hand, this made the listening experience more intense, allowing to concentrate on the aural information exclusively, but, on the other hand, it also imposed restrictions on people’s perception, as the syncretic listening and seeing experience became separated into constituents. Gestures can be considered as operating features of a person’s perception-action system. It presupposes significance of a meaning that involves more than just a physical movement. Movements can be subdivided into specific patterns and conceptualized. Conceptualized gestures are kept in people’s minds as single units, and the subdivision operations are carried out both by performers and the audience. Musical communication through gestures is therefore not about movement only, it should be viewed as structured interactions. For this research, solo percussion contemporary music performance will be analyzed. Overall, percussive music performance is extremely wide, and is accompanied by bright visual images provided by musicians themselves. From this perspective, observation over percussionists’ playing manner and it´s audience provides the researcher an opportunity to understand a narrative ability of music through musicians’ gestures. The quantitative research design divided in three experiments was chosen for the purpose of this study, which can be referred to as the description of the objective reality by using numbers in order to construct meaningful models reflecting various relationships between objects or phenomena. These numerical entities are not the reality itself, but a way of representing it. Moreover, the chosen experimental design gives an opportunity to not only establish the existence of certain effects of one variable on the other one, but also study the magnitude of these effects, considering the major two research questions: Is it possible to detect a percussive gesture’s narrative ? How does the percussive gesture influences the perception of musical narrative?Os gestos que produzem o som são reconhecidos pela maioria dos teóricos como um factor determinante da performance musical e da sua percepção. A conexão entre os gestos, sons e percepção de determinado discurso musical foi já abordada por um amplo número de cientistas, ainda que não haja um claro consenso quanto à medida em que essa conexão é fundamental ou quanto às operações cognitivas subjacentes à percepção de uma peça musical. O objetivo da interpretação em música consistirá não apenas em comunicar os sinais musicais que formam uma obra, mas também em transmitir a sucessão significativa de gestos, expressões e movimentos do corpo. Esta sucessão significativa, ou de outra forma exposto, a “narrativa do gesto” é considerada muito importante para o processo de condução de um público para a interpretação pretendida. O avanço da tecnologia neste estádio de desenvolvimento da sociedade, criou excelentes oportunidades para permitir a separação de atividades auditivas e visuais da música. A gravação e posterior difusão musical, permitiu que o público consumisse música sem ter que, para essa finalidade, presenciar um evento musical. Por um lado, esse fenómeno tornou a experiência de escuta mais frequente e porventura mais focada, permitindo ao ouvinte concentrar-se na informação auditiva exclusivamente. Mas, por outro lado, também impôs restrições a uma experiência musical sincrética com o ouvir e ouvir e ver, a separaram-se em constituintes dentro do fenómeno musical. Os gestos podem ser considerados características de funcionamento do sistema de percepção/ acção de um ser humano. Pressupõe isso a atribuição de expressão a um significado que envolve mais do que apenas um movimento físico. Os movimentos podem ser subdivididos em padrões específicos e conceptualizados. Estes gestos conceptualizados são mantidos como unidades singulares, e as operações de subdivisão significantes são levadas a cabo tanto pelos performers como pelo seu público. A comunicação musical através de gestos, não deve, portanto, ser olhada apenas sobre os aspectos do movimento, mas sim como uma interação estruturada e musicalmente contextualizada. Os processos descritos acima resultam em grande parte do ambiente de envolvimento do individuo ouvinte e dependem fortemente da sua singularidade e contexto cultural. Nem todos os movimentos poderão ser chamados gestos performativos para além daqueles aqueles cuja acção é de natureza intencionalmente expressiva ou inerente à produção de som. Nesta pesquisa, a performance de música contemporânea para percussão solo será analisada. De uma maneira geral, o desempenho dos percussionistas é, do ponto de vista visual, extremamente rico na formação de gestos. Nessa perspectiva, a observação de uma audiência sujeita à sua presença, com e sem contacto visual com a sua acção, fornece uma oportunidade de aproximação ao estudo do gesto e da sua narrativa, do ponto de vista da percepção do discurso musical. Um desenho de pesquisa quantitativa dividida em três experiências, foi o caminho escolhido para o presente estudo. Produziu-se uma descrição da realidade objectiva usando números, de modo a construir modelos significativos que pudessem reflectir as várias relações entre objectos ou fenómenos . Estas entidades numéricas não serão assim uma realidade em si, mas uma maneira possível de a representar . O processo de experimentação, dividido em três partes, dá-nos a oportunidade para perceber não só a existência dos efeitos de uma variável sobre a outra (visual e auditiva), mas também permite uma reflexão sobre a magnitude desses efeitos, tentando assim responder ás questões que levam a esta investigação: É possivel detectar uma narrativa no gesto percussivo? Como é que o gesto percussivo influencia a percepção do discurso musical

Contribution de la motivation dans les jeux sérieux

La motivation incite les apprenants à s’engager dans une activité et à persévérer dans son accomplissement afin d’atteindre un but. Dans les Systèmes Tutoriels Intelligents (STI), les études sur la motivation des apprenants possèdent trois manques importants : un manque de moyens objectifs et fiables pour évaluer cet état, un manque d’évaluation de rôles joués par les facteurs motivationnels conçus dans l’environnement d’apprentissage et un manque de stratégies d’interventions motivationnelles pour soutenir la motivation des apprenants. Dans cette thèse, nous nous intéressons à mieux comprendre l’état de la motivation des apprenant ainsi que les facteurs et stratégies motivationnels dans un environnement d’apprentissage captivant : les jeux sérieux. Dans une première étude, nous évaluons la motivation des apprenants par l’entremise d’un modèle théorique de la motivation (ARCS de Keller) et de données électro-physiologiques (la conductivité de la peau, le rythme cardiaque et l’activité cérébrale). Nous déterminons et évaluons aussi quelques situations ou stratégies favorisant la motivation dans l’environnement des jeux sérieux étudié. Dans une deuxième étude, nous développons un prototype de jeux sérieux intégrant – dans une première version – quelques éléments motivationnels issus de jeux vidéo et – dans une deuxième version – des stratégies motivationnelles d’un modèle théorique de la motivation. Nous espérons, avec une évaluation motivationnelle de notre prototype, soutenir les apprenants à atteindre des hauts niveaux de motivation, de persévérance et de performance.Motivation encourages learners to be engaged in an activity and to persevere in its accomplishment in order to achieve a goal. In Intelligent Tutoring Systems (ITS), different studies of learners’ motivation have showed three major lacks: a lack of objective and reliable means to assess this state, a lack of evaluation of the roles played by motivational factors developed by learning environments, and a lack of motivational interventions to support learners’ motivation. In this thesis, we are interested in understanding the state of motivation, as well as motivational factors and strategies in an exciting learning environment: serious games. First, we carry out an empirical study to assess learners’ motivation using Keller’s ARCS psychological model combined with electro-physiological recordings, namely skin conductance, heart rate, and brain activity. We also identify and evaluate different situations and strategies that promote motivation in a serious game environment. Second, we develop a serious game which has some motivational elements (version 1) as well as different motivational strategies (version 2). Our serious game intends to support learners to rich high levels of motivation, perseverance and performance. We conduct an empirical assessment of learners’ motivation during interaction with our serious game

Answering questions about archived, annotated meetings

Retrieving information from archived meetings is a new domain of information retrieval that has received increasing attention in the past few years. Search in spontaneous spoken conversations has been recognized as more difficult than text-based document retrieval because meeting discussions contain two levels of information: the content itself, i.e. what topics are discussed, but also the argumentation process, i.e. what conflicts are resolved and what decisions are made. To capture the richness of information in meetings, current research focuses on recording meetings in Smart-Rooms, transcribing meeting discussion into text and annotating discussion with semantic higher-level structures to allow for efficient access to the data. However, it is not yet clear what type of user interface is best suited for searching and browsing such archived, annotated meetings. Content-based retrieval with keyword search is too naive and does not take into account the semantic annotations on the data. The objective of this thesis is to assess the feasibility and usefulness of a natural language interface to meeting archives that allows users to ask complex questions about meetings and retrieve episodes of meeting discussions based on semantic annotations. The particular issues that we address are: the need of argumentative annotation to answer questions about meetings; the linguistic and domain-specific natural language understanding techniques required to interpret such questions; and the use of visual overviews of meeting annotations to guide users in formulating questions. To meet the outlined objectives, we have annotated meetings with argumentative structure and built a prototype of a natural language understanding engine that interprets questions based on those annotations. Further, we have performed two sets of user experiments to study what questions users ask when faced with a natural language interface to annotated meeting archives. For this, we used a simulation method called Wizard of Oz, to enable users to express questions in their own terms without being influenced by limitations in speech recognition technology. Our experimental results show that technically it is feasible to annotate meetings and implement a deep-linguistic NLU engine for questions about meetings, but in practice users do not consistently take advantage of these features. Instead they often search for keywords in meetings. When visual overviews of the available annotations are provided, users refer to those annotations in their questions, but the complexity of questions remains simple. Users search with a breadth-first approach, asking questions in sequence instead of a single complex question. We conclude that natural language interfaces to meeting archives are useful, but that more experimental work is needed to find ways to incent users to take advantage of the expressive power of natural language when asking questions about meetings