Feasibility study of an Integrated Program for Aerospace vehicle Design (IPAD). Volume 1B: Concise review

Reports on the design process, support of the design process, IPAD System design catalog of IPAD technical program elements, IPAD System development and operation, and IPAD benefits and impact are concisely reviewed. The approach used to define the design is described. Major activities performed during the product development cycle are identified. The computer system requirements necessary to support the design process are given as computational requirements of the host system, technical program elements and system features. The IPAD computer system design is presented as concepts, a functional description and an organizational diagram of its major components. The cost and schedules and a three phase plan for IPAD implementation are presented. The benefits and impact of IPAD technology are discussed

深層学習に基づく感情会話分析に関する研究

Owning the capability to express specific emotions by a chatbot during a conversation is one of the key parts of artificial intelligence, which has an intuitive and quantifiable impact on the improvement of chatbot’s usability and user satisfaction. Enabling machines to emotion recognition in conversation is challenging, mainly because the information in human dialogue innately conveys emotions by long-term experience, abundant knowledge, context, and the intricate patterns between the affective states. Recently, many studies on neural emotional conversational models have been conducted. However, enabling the chatbot to control what kind of emotion to respond to upon its own characters in conversation is still underexplored. At this stage, people are no longer satisfied with using a dialogue system to solve specific tasks, and are more eager to achieve spiritual communication. In the chat process, if the robot can perceive the user's emotions and can accurately process them, it can greatly enrich the content of the dialogue and make the user empathize. In the process of emotional dialogue, our ultimate goal is to make the machine understand human emotions and give matching responses. Based on these two points, this thesis explores and in-depth emotion recognition in conversation task and emotional dialogue generation task. In the past few years, although considerable progress has been made in emotional research in dialogue, there are still some difficulties and challenges due to the complex nature of human emotions. The key contributions in this thesis are summarized as below: (1) Researchers have paid more attention to enhancing natural language models with knowledge graphs these days, since knowledge graph has gained a lot of systematic knowledge. A large number of studies had shown that the introduction of external commonsense knowledge is very helpful to improve the characteristic information. We address the task of emotion recognition in conversations using external knowledge to enhance semantics. In this work, we employ an external knowledge graph ATOMIC to extract the knowledge sources. We proposed KES model, a new framework that incorporates different elements of external knowledge and conversational semantic role labeling, where build upon them to learn interactions between interlocutors participating in a conversation. The conversation is a sequence of coherent and orderly discourses. For neural networks, the capture of long-range context information is a weakness. We adopt Transformer a structure composed of self-attention and feed forward neural network, instead of the traditional RNN model, aiming at capturing remote context information. We design a self-attention layer specialized for enhanced semantic text features with external commonsense knowledge. Then, two different networks composed of LSTM are responsible for tracking individual internal state and context external state. In addition, the proposed model has experimented on three datasets in emotion detection in conversation. The experimental results show that our model outperforms the state-of-the-art approaches on most of the tested datasets. (2) We proposed an emotional dialogue model based on Seq2Seq, which is improved from three aspects: model input, encoder structure, and decoder structure, so that the model can generate responses with rich emotions, diversity, and context. In terms of model input, emotional information and location information are added based on word vectors. In terms of the encoder, the proposed model first encodes the current input and sentence sentiment to generate a semantic vector, and additionally encodes the context and sentence sentiment to generate a context vector, adding contextual information while ensuring the independence of the current input. On the decoder side, attention is used to calculate the weights of the two semantic vectors separately and then decode, to fully integrate the local emotional semantic information and the global emotional semantic information. We used seven objective evaluation indicators to evaluate the model's generation results, context similarity, response diversity, and emotional response. Experimental results show that the model can generate diverse responses with rich sentiment, contextual associations

マルチモーダル音声対話システムでの先進的コミュニケーションのためのユーザ状態推定

Tohoku University伊藤彰則課

Four Mode Based Dialogue Management with Modified POMDP Model

This thesis proposes a method to manage the interaction between the user and the system dynamically, through speech or text input which updates the user goals, select system actions and calculate rewards for each system response at each time-stamp. The main focus is made on the dialog manager, which decides how to continue the dialogue. We have used POMDP technique, as it maintains a belief distribution on the dialogue states based on the observations over the dialogue even in a noisy environment. Four contextual control modes are introduced in dialogue management for decision-making mechanism, and to keep track of machine behaviour for each dialogue state. The result obtained proves that our proposed framework has overcome the limitations of prior POMDP methods, and exactly understands the actual intention of the users within the available time, providing very interactive conversation between the user and the computer

Improved Intention Discovery with Classified Emotions in A Modified POMDP

Emotions are one of the most proactive topics in psychology, a basis of forceful conversation and divergence from the earliest philosophers and other thinkers to the present day. Human emotion classification using different machine learning techniques is an active area of research over the last decade. This investigation discusses a new approach for virtual agents to better understand and interact with the user. Our research focuses on deducing the belief state of a user who interacts with a single agent using recognized emotions from the text/speech based input. We built a customized decision tree with six primary states of emotions being recognized from different sets of inputs. The belief state at each given instance of time slice is inferred by drawing a belief network using the different sets of emotions and calculating state of belief using a POMDP (Partially Observable Markov Decision Process) based solver. Hence the existing POMDP model is customized in order to incorporate emotion as observations for finding the possible user intentions. This helps to overcome the limitations of the present methods to better recognize the belief state. As well, the new approach allows us to analyze human emotional behaviour in indefinite environments and helps to generate an effective interaction between the human and the computer

MULTI-MODAL TASK INSTRUCTIONS TO ROBOTS BY NAIVE USERS

This thesis presents a theoretical framework for the design of user-programmable robots. The objective of the work is to investigate multi-modal unconstrained natural instructions given to robots in order to design a learning robot. A corpus-centred approach is used to design an agent that can reason, learn and interact with a human in a natural unconstrained way. The corpus-centred design approach is formalised and developed in detail. It requires the developer to record a human during interaction and analyse the recordings to find instruction primitives. These are then implemented into a robot. The focus of this work has been on how to combine speech and gesture using rules extracted from the analysis of a corpus. A multi-modal integration algorithm is presented, that can use timing and semantics to group, match and unify gesture and language. The algorithm always achieves correct pairings on a corpus and initiates questions to the user in ambiguous cases or missing information. The domain of card games has been investigated, because of its variety of games which are rich in rules and contain sequences. A further focus of the work is on the translation of rule-based instructions. Most multi-modal interfaces to date have only considered sequential instructions. The combination of frame-based reasoning, a knowledge base organised as an ontology and a problem solver engine is used to store these rules. The understanding of rule instructions, which contain conditional and imaginary situations require an agent with complex reasoning capabilities. A test system of the agent implementation is also described. Tests to confirm the implementation by playing back the corpus are presented. Furthermore, deployment test results with the implemented agent and human subjects are presented and discussed. The tests showed that the rate of errors that are due to the sentences not being defined in the grammar does not decrease by an acceptable rate when new grammar is introduced. This was particularly the case for complex verbal rule instructions which have a large variety of being expressed

Decision fusion for multi-modal person authentication.

Hui Pak Sum Henry.Thesis (M.Phil.)--Chinese University of Hong Kong, 2006.Includes bibliographical references (leaves [147]-152).Abstracts in English and Chinese.Chapter 1. --- Introduction --- p.1Chapter 1.1. --- Objectives --- p.4Chapter 1.2. --- Thesis Outline --- p.5Chapter 2. --- Background --- p.6Chapter 2.1. --- User Authentication Systems --- p.6Chapter 2.2. --- Biometric Authentication --- p.9Chapter 2.2.1. --- Speaker Verification System --- p.9Chapter 2.2.2. --- Face Verification System --- p.10Chapter 2.2.3. --- Fingerprint Verification System --- p.11Chapter 2.3. --- Verbal Information Verification (VIV) --- p.12Chapter 2.4. --- Combining SV and VIV --- p.15Chapter 2.5. --- Biometric Decision Fusion Techniques --- p.17Chapter 2.6. --- Fuzzy Logic --- p.20Chapter 2.6.1. --- Fuzzy Membership Function and Fuzzy Set --- p.21Chapter 2.6.2. --- Fuzzy Operators --- p.22Chapter 2.6.3. --- Fuzzy Rules --- p.22Chapter 2.6.4. --- Defuzzification --- p.23Chapter 2.6.5. --- Advantage of Using Fuzzy Logic in Biometric Fusion --- p.23Chapter 2.7. --- Chapter Summary --- p.25Chapter 3. --- Experimental Data --- p.26Chapter 3.1. --- Data for Multi-biometric Fusion --- p.26Chapter 3.1.1. --- Speech Utterances --- p.30Chapter 3.1.2. --- Face Movement Video Frames --- p.31Chapter 3.1.3. --- Fingerprint Images --- p.32Chapter 3.2. --- Data for Speech Authentication Fusion --- p.33Chapter 3.2.1. --- SV Training Data for Speaker Model --- p.34Chapter 3.2.2. --- VIV Training Data for Speaker Independent Model --- p.34Chapter 3.2.3. --- Validation Data --- p.34Chapter 3.3. --- Chapter Summary --- p.36Chapter 4. --- Authentication Modules --- p.37Chapter 4.1. --- Biometric Authentication --- p.38Chapter 4.1.1. --- Speaker Verification --- p.38Chapter 4.1.2. --- Face Verification --- p.38Chapter 4.1.3. --- Fingerprint Verification --- p.39Chapter 4.1.4. --- Individual Biometric Performance --- p.39Chapter 4.2. --- Verbal Information Verification (VIV) --- p.42Chapter 4.3. --- Chapter Summary --- p.44Chapter 5. --- Weighted Average Fusion for Multi-Modal Biometrics --- p.46Chapter 5.1. --- Experimental Setup and Results --- p.46Chapter 5.2. --- Analysis of Weighted Average Fusion Results --- p.48Chapter 5.3. --- Chapter Summary --- p.59Chapter 6. --- Fully Adaptive Fuzzy Logic Decision Fusion Framework --- p.61Chapter 6.1. --- Factors Considered in the Estimation of Biometric Sample Quality --- p.62Chapter 6.1.1. --- Factors for Speech --- p.63Chapter 6.1.2. --- Factors for Face --- p.65Chapter 6.1.3. --- Factors for Fingerprint --- p.70Chapter 6.2. --- Fuzzy Logic Decision Fusion Framework --- p.76Chapter 6.2.1. --- Speech Fuzzy Sets --- p.77Chapter 6.2.2. --- Face Fuzzy Sets --- p.79Chapter 6.2.3. --- Fingerprint Fuzzy Sets --- p.80Chapter 6.2.4. --- Output Fuzzy Sets --- p.81Chapter 6.2.5. --- Fuzzy Rules and Other Information --- p.83Chapter 6.3. --- Experimental Setup and Results --- p.84Chapter 6.4. --- Comparison Between Weighted Average and Fuzzy Logic Decision Fusion --- p.86Chapter 6.5. --- Chapter Summary --- p.95Chapter 7. --- Factors Affecting VIV Performance --- p.97Chapter 7.1. --- Factors from Verbal Messages --- p.99Chapter 7.1.1. --- Number of Distinct-Unique Responses --- p.99Chapter 7.1.2. --- Distribution of Distinct-Unique Responses --- p.101Chapter 7.1.3. --- Inter-person Lexical Choice Variations --- p.103Chapter 7.1.4. --- Intra-person Lexical Choice Variations --- p.106Chapter 7.2. --- Factors from Utterance Verification --- p.108Chapter 7.2.1. --- Thresholding --- p.109Chapter 7.2.2. --- Background Noise --- p.113Chapter 7.3. --- VIV Weight Estimation Using PDP --- p.115Chapter 7.4. --- Chapter Summary --- p.119Chapter 8. --- Adaptive Fusion for SV and VIV --- p.121Chapter 8.1. --- Weighted Average fusion of SV and VIV --- p.122Chapter 8.1.1. --- Scores Normalization --- p.123Chapter 8.1.2. --- Experimental Setup --- p.123Chapter 8.2. --- Adaptive Fusion for SV and VIV --- p.124Chapter 8.2.1. --- Components of Adaptive Fusion --- p.126Chapter 8.2.2. --- Three Categories Design --- p.129Chapter 8.2.3. --- Fusion Strategy for Each Category --- p.132Chapter 8.2.4. --- SV Driven Approach --- p.133Chapter 8.3. --- SV and Fixed-Pass Phrase VIV Fusion Results --- p.133Chapter 8.4. --- SV and Key-Pass Phrase VIV Fusion Results --- p.136Chapter 8.5. --- Chapter Summary --- p.141Chapter 9. --- Conclusions and Future Work --- p.143Chapter 9.1. --- Conclusions --- p.143Chapter 9.2. --- Future Work --- p.145Bibliography --- p.147Appendix A Detail of BSC Speech --- p.153Appendix B Fuzzy Rules for Multimodal Biometric Fusion --- p.155Appendix C Full Example for Multimodal Biometrics Fusion --- p.157Appendix DReason for Having a Flat Error Surface --- p.161Appendix E Reason for Having a Relative Peak Point in the Middle of the Error Surface --- p.164Appendix F Illustration on Fuzzy Logic Weight Estimation --- p.166Appendix GExamples for SV and Key-Pass Phrase VIV Fusion --- p.17

Communicative humanoids : a computational model of psychosocial dialogue skills

Thesis (Ph. D.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1996.Includes bibliographical references (p. [223]-238).Kristinn Rúnar Thórisson.Ph.D