A Cooperative Negotiation Protocol for Physiological Model Combination

The global model of a complex phenomenon can emerge from the cooperative negotiation of agents embedding local partial models of the phenomenon. We adopted this approach to model complex physiological phenomena, such as those related to the metabolism of glucose-insulin and to the determination of the heart rate (pacing). In this paper we formally describe and analyze the properties of a cooperative negotiation protocol we developed to allow the agents to produce a global coherent model of a physiological phenomenon. We concentrate on the study of the conditions under which an agreement is guaranteed to be reached. We also show details of an application concerning the pacing problem

Advances in Reinforcement Learning

Reinforcement Learning (RL) is a very dynamic area in terms of theory and application. This book brings together many different aspects of the current research on several fields associated to RL which has been growing rapidly, producing a wide variety of learning algorithms for different applications. Based on 24 Chapters, it covers a very broad variety of topics in RL and their application in autonomous systems. A set of chapters in this book provide a general overview of RL while other chapters focus mostly on the applications of RL paradigms: Game Theory, Multi-Agent Theory, Robotic, Networking Technologies, Vehicular Navigation, Medicine and Industrial Logistic

Multi-agent model of hepatitis C virus infection

Objectives: The objective of this study is to design a method for modeling hepatitis C virus (HCV) infection using multi-agent simulation and to verify it in practice. Methods and materials: In this paper, first, the modeling of HCV infection using a multi-agent system is compared with the most commonly used model type, which is based on differential equations. Then, the implementation and results of the model using a multi-agent simulation is presented. To find the values of the parameters used in the model, a method using inverted simulation flow and genetic algorithm is proposed. All of the data regarding HCV infection are taken from the paper describing the model based on the differential equation to which the proposed method is compared. Results: Important advantages of the proposed method are noted and demonstrated; these include flexibility, clarity, re-usability and the possibility to model more complex dependencies. Then, the simulation framework that uses the proposed approach is successfully implemented in C++ and is verified by comparing it to the approach based on differential equations. The verification proves that an objective function that performs the best is the function that minimizes the maximal differences in the data. Finally, an analysis of one of the already known models is performed, and it is proved that it incorrectly models a decay in the hepatocytes number by 40%. Conclusions: The proposed method has many advantages in comparison to the currently used model types and can be used successfully for analyzing HCV infection. With almost no modifications, it can also be used for other types of viral infections

Closed-Loop Quantitative Verification of Rate-Adaptive Pacemakers

Rate-adaptive pacemakers are cardiac devices able to automatically adjust the pacing rate in patients with chronotropic incompetence, i.e. whose heart is unable to provide an adequate rate at increasing levels of physical, mental or emotional activity. These devices work by processing data from physiological sensors in order to detect the patient’s activity and update the pacing rate accordingly. Rate-adaptation parameters depend on many patient-specific factors, and effective personalisation of such treatments can only be achieved through extensive exercise testing, which is normally intolerable for a cardiac patient. In this work, we introduce a data-driven and model-based approach for the automated verification of rate-adaptive pacemakers and formal analysis of personalised treatments. To this purpose, we develop a novel dual-sensor pacemaker model where the adaptive rate is computed by blending information from an accelerometer, and a metabolic sensor based on the QT interval. Our approach enables personalisation through the estimation of heart model parameters from patient data (electrocardiogram), and closed-loop analysis through the online generation of synthetic, model-based QT intervals and acceleration signals. In addition to personalisation, we also support the derivation of models able to account for the varied characteristics of a virtual patient population, thus enabling safety verification of the device. To capture the probabilistic and non-linear dynamics of the heart, we define a probabilistic extension of timed I/O automata with data and employ statistical model checking for quantitative verification of rate modulation. We evaluate our rate-adaptive pacemaker design on three subjects and a pool of virtual patients, demonstrating the potential of our approach to provide rigorous, quantitative insights into the closed-loop behaviour of the device under different exercise levels and heart conditions

Affective reactions towards socially interactive agents and their computational modeling

Over the past 30 years, researchers have studied human reactions towards machines applying the Computers Are Social Actors paradigm, which contrasts reactions towards computers with reactions towards humans. The last 30 years have also seen improvements in technology that have led to tremendous changes in computer interfaces and the development of Socially Interactive Agents. This raises the question of how humans react to Socially Interactive Agents. To answer these questions, knowledge from several disciplines is required, which is why this interdisciplinary dissertation is positioned within psychology and computer science. It aims to investigate affective reactions to Socially Interactive Agents and how these can be modeled computationally. Therefore, after a general introduction and background, this thesis first provides an overview of the Socially Interactive Agent system used in this work. Second, it presents a study comparing a human and a virtual job interviewer, which shows that both interviewers induce shame in participants to the same extent. Thirdly, it reports on a study investigating obedience towards Socially Interactive Agents. The results indicate that participants obey human and virtual instructors in similar ways. Furthermore, both types of instructors evoke feelings of stress and shame to the same extent. Fourth, a stress management training using biofeedback with a Socially Interactive Agent is presented. The study shows that a virtual trainer can teach coping techniques for emotionally challenging social situations. Fifth, it introduces MARSSI, a computational model of user affect. The evaluation of the model shows that it is possible to relate sequences of social signals to affective reactions, taking into account emotion regulation processes. Finally, the Deep method is proposed as a starting point for deeper computational modeling of internal emotions. The method combines social signals, verbalized introspection information, context information, and theory-driven knowledge. An exemplary application to the emotion shame and a schematic dynamic Bayesian network for its modeling are illustrated. Overall, this thesis provides evidence that human reactions towards Socially Interactive Agents are very similar to those towards humans, and that it is possible to model these reactions computationally.In den letzten 30 Jahren haben Forschende menschliche Reaktionen auf Maschinen untersucht und dabei das “Computer sind soziale Akteure”-Paradigma genutzt, in dem Reaktionen auf Computer mit denen auf Menschen verglichen werden. In den letzten 30 Jahren hat sich ebenfalls die Technologie weiterentwickelt, was zu einer enormen Veränderung der Computerschnittstellen und der Entwicklung von sozial interaktiven Agenten geführt hat. Dies wirft Fragen zu menschlichen Reaktionen auf sozial interaktive Agenten auf. Um diese Fragen zu beantworten, ist Wissen aus mehreren Disziplinen erforderlich, weshalb diese interdisziplinäre Dissertation innerhalb der Psychologie und Informatik angesiedelt ist. Sie zielt darauf ab, affektive Reaktionen auf sozial interaktive Agenten zu untersuchen und zu erforschen, wie diese computational modelliert werden können. Nach einer allgemeinen Einführung in das Thema gibt diese Arbeit daher, erstens, einen Überblick über das Agentensystem, das in der Arbeit verwendet wird. Zweitens wird eine Studie vorgestellt, in der eine menschliche und eine virtuelle Jobinterviewerin miteinander verglichen werden, wobei sich zeigt, dass beide Interviewerinnen bei den Versuchsteilnehmenden Schamgefühle in gleichem Maße auslösen. Drittens wird eine Studie berichtet, in der Gehorsam gegenüber sozial interaktiven Agenten untersucht wird. Die Ergebnisse deuten darauf hin, dass Versuchsteilnehmende sowohl menschlichen als auch virtuellen Anleiterinnen ähnlich gehorchen. Darüber hinaus werden durch beide Instruktorinnen gleiche Maße von Stress und Scham hervorgerufen. Viertens wird ein Biofeedback-Stressmanagementtraining mit einer sozial interaktiven Agentin vorgestellt. Die Studie zeigt, dass die virtuelle Trainerin Techniken zur Bewältigung von emotional herausfordernden sozialen Situationen vermitteln kann. Fünftens wird MARSSI, ein computergestütztes Modell des Nutzeraffekts, vorgestellt. Die Evaluation des Modells zeigt, dass es möglich ist, Sequenzen von sozialen Signalen mit affektiven Reaktionen unter Berücksichtigung von Emotionsregulationsprozessen in Beziehung zu setzen. Als letztes wird die Deep-Methode als Ausgangspunkt für eine tiefer gehende computergestützte Modellierung von internen Emotionen vorgestellt. Die Methode kombiniert soziale Signale, verbalisierte Introspektion, Kontextinformationen und theoriegeleitetes Wissen. Eine beispielhafte Anwendung auf die Emotion Scham und ein schematisches dynamisches Bayes’sches Netz zu deren Modellierung werden dargestellt. Insgesamt liefert diese Arbeit Hinweise darauf, dass menschliche Reaktionen auf sozial interaktive Agenten den Reaktionen auf Menschen sehr ähnlich sind und dass es möglich ist diese menschlichen Reaktion computational zu modellieren.Deutsche Forschungsgesellschaf

The strategic impacts of Intelligent Automation for knowledge and service work : An interdisciplinary review

We would like to thank Professor Jarvenpaa and the review team for all the constructive comments and suggestions that were most helpful in revising the paper and in offering a stronger contribution. We would also like to thank Professor Guy Fitzgerald for his constructive comments on earlier versions of the paper. This study was funded by the Chartered Institute of Professional Development (CIPD). The views expressed are those of the authors and not necessarily those of the CIPD.Peer reviewedPublisher PD

Biosignal controlled recommendation in entertainment systems

With the explosive growth of the entertainment contents and the ubiquitous access of them via fixed or mobile computing devices, recommendation systems become essential tools to help the user to find the right entertainment at the right time and location. I envision that by integrating the bio signal input into the recommendation process, it will help the users not only to find interesting contents, but also to increase one’s comfort level by taking into account the biosginal feedback from the users. The goal of this project was to develop a biosignal controlled entertainment recommendation system that increases the user’s comfort level by reducing the level of stress. As the starting point, this project aims to contribute to the field of recommendation systems with two points. The first is the mechanism of embedding the biosignal non-intrusively into the recommendation process. The second is the strategy of the biosignal controlled recommendation to reduce stress. Heart rate controlled in-flight music recommendation is chosen as its application domain. The hypothesis of this application is that, the passenger's heart rate deviates from the normal due to unusual long haul flight cabin environment. By properly designing a music recommendation system to recommend heart rate controlled personalized music playlists to the passenger, the passengers' heart rate can be uplifted, down-lifted back to normal or kept within normal, thus their stress can be reduced. Four research questions have been formulated based on this hypothesis. After the literature study, the project went mainly through three phases: framework design, system implementation and user evaluation to answer these research questions. During the framework design phase, the heart rate was firstly modeled as the states of bradycardia, normal and tachycardia. The objective of the framework is that, if the user's heart rate is higher or lower than the normal heart rate, the system recommends a personalized music playlist accordingly to transfer the user’s heart rate back to normal, otherwise to keep it at normal. The adaptive framework integrates the concepts of context adaptive systems, user profiling, and the methods of using music to adjust the heart rate in a feedback control system. In the feedback loop, the playlists were composed using a Markov decision process. Yet, the framework allows the user to reject the recommendations and to manually select the favorite music items. During this process, the system logs the interactions between the user and the system for later learning the user’s latest music preferences. The designed framework was then implemented with platform independent software architecture. The architecture has five abstraction levels. The lowest resource level contains the music source, the heart rate sensors and the user profile information. The second layer is for resource management. In this layer are the manager components to manage the resources from the first layer and to modulate the access from upper layers to these resources. The third layer is the database, acting as a data repository. The fourth layer is for the adaptive control, which includes the user feedback log, the inference engine and the preference learning component. The top layer is the user interface. In this architecture, the layers and the components in the layers are loosely coupled, which ensures the flexibility. The implemented system was used in the user experiments to validate the hypothesis. The experiments simulated the long haul flights from Amsterdam to Shanghai with the same time schedule as the KLM flights. Twelve subjects were invited to participate in the experiments. Six were allocated to the controlled group and others were allocated to the treatment group. In addition to a normal entertainment system for the control group, the treatment group was also provided with the heart rate controlled music recommendation system. The experiments results validated the hypothesis and answered the research questions. The passenger's heart rate deviates from normal. In our user experiments, the passenger's heart rate was in the bradycardia state 24.6% of time and was in the tachycardia state 7.3% of time. The recommended uplifting music reduces the average bradycardia state duration from 14.78 seconds in the control group to 6.86 seconds in the treatment group. The recommended keeping music increases the average normal state duration from 24.66 seconds in the control group to 29.79 seconds in the treatment group. The recommended down-lifting music reduces the average tachycardia state duration from 13.89 seconds in the control group to 6.53 seconds in the treatment group. Compared to the control group, the stress of the treatment group has been reduced significantly

Model Checking Stochastic Systems in PAT

Ph.DDOCTOR OF PHILOSOPH

Statically-analyzed stream monitoring for cyber-physical Systems

Cyber-physical systems are digital systems interacting with the physical world. Even though this induces an inherent complexity, they are responsible for safety-critical tasks like governing nuclear power plants or controlling autonomous vehicles. To preserve trust into the safety of such systems, this thesis presents a runtime verification approach designed to generate trustworthy monitors from a formal specification. These monitors are responsible for observing the cyber-physical system during runtime and ensuring its safety. As underlying language, I present the asynchronous real-time specification language RTLola. It contains primitives for arithmetic properties and grants precise control over the timing of the monitor. With this, it enables specifiers to express properties relevant to cyber-physical systems. The thesis further presents a static analysis that identifies inconsistencies in the specification and provides insights into the dynamic behavior of the monitor. As a result, the resource consumption of the monitor becomes predictable. The generation of the monitor produces either a hardware description synthesizable onto programmable hardware, or Rust code with verification annotation. These annotations allow for proving the correctness of the monitor with respect to the semantics of RTLola. Last, I present the construction of a conservative hybrid model of the underlying system using information extracted from the specification. This model enables further verification steps.Cyber-physische Systeme sind digitale Systeme, die mit der physischen Welt interagieren. Obwohl das zu einer inhärenten Komplexität führt, sind sie verantwortlich für sicherheitskritische Aufgaben wie der Steuerung von Kernkraftwerken oder autonomen Fahrzeugen. Umdas Vertrauen in deren Sicherheit zu wahren, präsentiert diese Doktorarbeit einen Ansatz zur Laufzeitverifikation, konzipiert, um vertrauenswürdige Monitore aus einer formalen Spezifikation zu generieren. Diese Monitore sind dafür verantwortlich, das cyber-physische System zur Laufzeit zu überwachen und dessen Sicherheit zu gewährleisten. Als zugrundeliegende Sprache präsentiere ich die asynchrone Echtzeit-Spezifikationssprache RTLola. Sie enthält Primitiven für arithmetische Eigenschaften und gewährt präzise Kontrolle über das Timing des Monitors. Damit wird es Spezifizierenden ermöglicht Eigenschaften auszudrücken, die für Cyber-physische Systeme relevant sind. Weiterhin präsentiert diese Doktorarbeit eine statische Analyse, die Unstimmigkeiten in der Spezifikation identifiziert und Einblicke in das dynamische Verhalten des Monitors liefert. Aufgrund dessen wird der Ressourcenverbrauch des Monitors vorhersehbar. Die Generierung des Monitors erzeugt entweder eine Hardwarebeschreibung, die auf programmierbarer Hardware synthetisiert werden kann, oder Rust Code mit Verifikationsannotationen. Diese Annotationen erlauben es, die Korrektheit des Monitors bezogen auf die Semantik von RTLola zu beweisen. Abschließend präsentiere ich die Konstruktion von einem konservativen hybriden Modell des zugrundeliegenden Systems anhand von Informationen, die aus der Spezifikation gewonnen wurden. Dieses Modell ermöglicht weitere Verifikationsschritte

Agent-based management of clinical guidelines

Les guies de pràctica clínica (GPC) contenen un conjunt d'accions i dades que ajuden a un metge a prendre decisions sobre el diagnòstic, tractament o qualsevol altre procediment a un pacient i sobre una determinada malaltia. És conegut que l'adopció d'aquestes guies en la vida diària pot millorar l'assistència mèdica als pacients, pel fet que s'estandarditzen les pràctiques. Sistemes computeritzats que utilitzen GPC poden constituir part de sistemes d'ajut a la presa de decisions més complexos amb la finalitat de proporcionar el coneixement adequat a la persona adequada, en un format correcte i en el moment precís. L'automatització de l'execució de les GPC és el primer pas per la seva implantació en els centres mèdics.Per aconseguir aquesta implantació final, hi ha diferents passos que cal solucionar com per exemple, l'adquisició i representació de les GPC, la seva verificació formal, i finalment la seva execució. Aquesta Tesi està dirigida en l'execució de GPC i proposa la implementació d'un sistema multi-agent. En aquest sistema els diferents actors dels centres mèdics coordinen les seves activitats seguint un pla global determinat per una GPC. Un dels principals problemes de qualsevol sistema que treballa en l'àmbit mèdic és el tractament del coneixement. En aquest cas s'han hagut de tractar termes mèdics i organitzatius, que s'ha resolt amb la implementació de diferents ontologies. La separació de la representació del coneixement del seu ús és intencionada i permet que el sistema d'execució de GPC sigui fàcilment adaptable a les circumstàncies concretes dels centres, on varien el personal i els recursos disponibles.En paral·lel a l'execució de GPC, el sistema proposat manega preferències del pacient per tal d'implementar serveis adaptats al pacient. En aquesta àrea concretament, a) s'han definit un conjunt de criteris, b) aquesta informació forma part del perfil de l'usuari i serveix per ordenar les propostes que el sistema li proposa, i c) un algoritme no supervisat d'aprenentatge permet adaptar les preferències del pacient segons triï.Finalment, algunes idees d'aquesta Tesi actualment s'estan aplicant en dos projectes de recerca. Per una banda, l'execució distribuïda de GPC, i per altra banda, la representació del coneixement mèdic i organitzatiu utilitzant ontologies.Clinical guidelines (CGs) contain a set of directions or principles to assist the health care practitioner with patient care decisions about appropriate diagnostic, therapeutic, or other clinical procedures for specific clinical circumstances. It is widely accepted that the adoption of guideline-execution engines in daily practice would improve the patient care, by standardising the care procedures. Guideline-based systems can constitute part of a knowledge-based decision support system in order to deliver the right knowledge to the right people in the right form at the right time. The automation of the guideline execution process is a basic step towards its widespread use in medical centres.To achieve this general goal, different topics should be tackled, such as the acquisition of clinical guidelines, its formal verification, and finally its execution. This dissertation focuses on the execution of CGs and proposes the implementation of an agent-based platform in which the actors involved in health care coordinate their activities to perform the complex task of guideline enactment. The management of medical and organizational knowledge, and the formal representation of the CGs, are two knowledge-related topics addressed in this dissertation and tackled through the design of several application ontologies. The separation of the knowledge from its use is fully intentioned, and allows the CG execution engine to be easily customisable to different medical centres with varying personnel and resources.In parallel with the execution of CGs, the system handles citizen's preferences and uses them to implement patient-centred services. With respect this issue, the following tasks have been developed: a) definition of the user's criteria, b) use of the patient's profile to rank the alternatives presented to him, c) implementation of an unsupervised learning method to adapt dynamically and automatically the user's profile.Finally, several ideas of this dissertation are being directly applied in two ongoing funded research projects, including the agent-based execution of CGs and the ontological management of medical and organizational knowledge