An agent-based approach to immune modelling

This study focuses on trying to understand why the range of experience with respect to HIV infection is so diverse, especially as regards to the latency period. The challenge is to determine what assumptions can be made about the nature of the experience of antigenic invasion and diversity that can be modelled, tested and argued plausibly. To investigate this, an agent-based approach is used to extract high-level behaviour which cannot be described analytically from the set of interaction rules at the cellular level. A prototype model encompasses local variation in baseline properties contributing to the individual disease experience and is included in a network which mimics the chain of lymphatic nodes. Dealing with massively multi-agent systems requires major computational efforts. However, parallelisation methods are a natural consequence and advantage of the multi-agent approach. These are implemented using the MPI library

HIV modelling - parallel implementation strategies

We report on the development of a model to understand why the range of experience with respect to HIV infection is so diverse, especially with respect to the latency period. To investigate this, an agent-based approach is used to extract highlevel behaviour which cannot be described analytically from the set of interaction rules at the cellular level. A network of independent matrices mimics the chain of lymph nodes. Dealing with massively multi-agent systems requires major computational effort. However, parallelisation methods are a natural consequence and advantage of the multi-agent approach and, using the MPI library, are here implemented, tested and optimized. Our current focus is on the various implementations of the data transfer across the network. Three communications strategies are proposed and tested, showing that the most efficient approach is communication based on the natural lymph-network connectivity

Agent-Based Medical Diagnosis Systems

Medical diagnostics elaboration many times is a distributed and cooperative work, which involves more medical human specialists and different medical systems. Recent results described in the literature prove that medical diagnosis problems can be solved efficiently by large-scale medical multi-agent systems. Cooperative diagnosing of medical diagnosis problems by large-scale multi-agent systems makes the diagnoses elaborations easier and may increase the accuracy of elaborated diagnostics. The purpose of the study described in this paper consists in the development of a novel large-scale hybrid medical diagnosis system called LMDS. The LMDS system is composed from physicians, medical expert system agents developed in our previous works and medical ICMA agents. Medical ICMA agents represent a novel class of agents with the ICMA architecture developed in our previous works, endowed with medical diagnosis capability. The main novelty of the LMDS system consists in the novel classes of agent members of the system and the manner in which the members of the system contribute to the problems solving. Each diagnostics can be elaborated cooperatively by more members of the system. The diagnosis system can solve difficult medical diagnosis problems whose solving must be discovered cooperatively by the members of the system. Many difficult medical problem solving requires medical knowledge that cannot be detained by a single physician or a medical computational system. Simulations prove the correctness in operation of the LMDS system

Scaling reinforcement learning to the unconstrained multi-agent domain

Reinforcement learning is a machine learning technique designed to mimic the way animals learn by receiving rewards and punishment. It is designed to train intelligent agents when very little is known about the agent’s environment, and consequently the agent’s designer is unable to hand-craft an appropriate policy. Using reinforcement learning, the agent’s designer can merely give reward to the agent when it does something right, and the algorithm will craft an appropriate policy automatically. In many situations it is desirable to use this technique to train systems of agents (for example, to train robots to play RoboCup soccer in a coordinated fashion). Unfortunately, several significant computational issues occur when using this technique to train systems of agents. This dissertation introduces a suite of techniques that overcome many of these difficulties in various common situations. First, we show how multi-agent reinforcement learning can be made more tractable by forming coalitions out of the agents, and training each coalition separately. Coalitions are formed by using information-theoretic techniques, and we find that by using a coalition-based approach, the computational complexity of reinforcement-learning can be made linear in the total system agent count. Next we look at ways to integrate domain knowledge into the reinforcement learning process, and how this can signifi-cantly improve the policy quality in multi-agent situations. Specifically, we find that integrating domain knowledge into a reinforcement learning process can overcome training data deficiencies and allow the learner to converge to acceptable solutions when lack of training data would have prevented such convergence without domain knowledge. We then show how to train policies over continuous action spaces, which can reduce problem complexity for domains that require continuous action spaces (analog controllers) by eliminating the need to finely discretize the action space. Finally, we look at ways to perform reinforcement learning on modern GPUs and show how by doing this we can tackle significantly larger problems. We find that by offloading some of the RL computation to the GPU, we can achieve almost a 4.5 speedup factor in the total training process

Extensible Java based agent framework

Agentska paradigma predstavlja najprirodniji i najdosledniji postojeći pristup implementaciji distribuiranih sistema. Uz pomoć agenata moguće je u potpunosti realizovati koncept distribuiranih softverskih komponenti, koje će, osim rešenja zadatka na distribuiranom nivou, pružiti i određenu količinu autonomnosti i inteligencije da bi se zadati cilj ostvario. Agentsko okruženje predstavlja programsko okruženje koje upravlja životnim tokom agenata i obezbeđuje mu sve potrebne mehanizme za realizaciju zadatka. U ovoj doktorskoj disertaciji predložen je model agentskog okruženja baziran na tehnologiji distribuiranih komponenti, koji podržava FIPA specifikaciju i sledeće koncepte: razmenu poruka, mobilnost agenata, sigurnosne mehanizme i direktorijume agenata i servisa. Model agentskog okruženja je implementiran u J2EE tehnologiji. Podržan je sistem plug-in-ova za sve bitne komponente agentskog okruženja (menadžere). Modelovan je i implementiran koncept mobilnih zadataka. Dat je model i implementacija sistema međusobnog uređenja odnosa agentskih centara. Predloženo rešenje agentskog okruženja verifikovano je na bibliotečkom informacionom sistemu BISIS. Verifikacija je izvršena na sledećim agentskim zadacima: pretraživanje bibliotečke mreže, ocenjivanje kvaliteta zapisa i inteligentna raspodela opterećenja.Agent technology is one of the most consistent approaches to the distributed computing implementation. Agents can be used to fully implement distributed software component concept. Agents can solve distributed problems utilizing certain degree of autonomy and intelligence. Agent framework represents programming environment that controls agent life cycle and provides all necessary mechanisms for task execution. The subject of the dissertation is formal specification of an agent framework based on distributed component technology. This framework supports FIPA specification and following concepts: message interchange, agent mobility, security and agent and service directory. Agent framework is implemented in J2EE technology. Plug-in system is designed for all key elements of agent framework. Mobile tasks were specified and implemented. Also, inter-facilitator connectivity mechanism is specified and implemented. The framework is verified by a case study on the library information system BISIS. Following agent tasks were performed: library network search, library record quality estimation and intelligent load balansing.

Extensible Java based agent framework

Agentska paradigma predstavlja najprirodniji i najdosledniji postojeći pristup implementaciji distribuiranih sistema. Uz pomoć agenata moguće je u potpunosti realizovati koncept distribuiranih softverskih komponenti, koje će, osim rešenja zadatka na distribuiranom nivou, pružiti i određenu količinu autonomnosti i inteligencije da bi se zadati cilj ostvario. Agentsko okruženje predstavlja programsko okruženje koje upravlja životnim tokom agenata i obezbeđuje mu sve potrebne mehanizme za realizaciju zadatka. U ovoj doktorskoj disertaciji predložen je model agentskog okruženja baziran na tehnologiji distribuiranih komponenti, koji podržava FIPA specifikaciju i sledeće koncepte: razmenu poruka, mobilnost agenata, sigurnosne mehanizme i direktorijume agenata i servisa. Model agentskog okruženja je implementiran u J2EE tehnologiji. Podržan je sistem plug-in-ova za sve bitne komponente agentskog okruženja (menadžere). Modelovan je i implementiran koncept mobilnih zadataka. Dat je model i implementacija sistema međusobnog uređenja odnosa agentskih centara. Predloženo rešenje agentskog okruženja verifikovano je na bibliotečkom informacionom sistemu BISIS. Verifikacija je izvršena na sledećim agentskim zadacima: pretraživanje bibliotečke mreže, ocenjivanje kvaliteta zapisa i inteligentna raspodela opterećenja.Agent technology is one of the most consistent approaches to the distributed computing implementation. Agents can be used to fully implement distributed software component concept. Agents can solve distributed problems utilizing certain degree of autonomy and intelligence. Agent framework represents programming environment that controls agent life cycle and provides all necessary mechanisms for task execution. The subject of the dissertation is formal specification of an agent framework based on distributed component technology. This framework supports FIPA specification and following concepts: message interchange, agent mobility, security and agent and service directory. Agent framework is implemented in J2EE technology. Plug-in system is designed for all key elements of agent framework. Mobile tasks were specified and implemented. Also, inter-facilitator connectivity mechanism is specified and implemented. The framework is verified by a case study on the library information system BISIS. Following agent tasks were performed: library network search, library record quality estimation and intelligent load balansing.

The exploration of unknown environments by affective agents

Tese de doutoramento em Engenharia Informática apresentada à Fac. de Ciências e Tecnologia de CoimbraIn this thesis, we study the problem of the exploration of unknown environments populated with entities by affective autonomous agents. The goal of these agents is twofold: (i) the acquisition of maps of the environment – metric maps – to be stored in memory, where the cells occupied by the entities that populate that environment are represented; (ii) the construction of models of those entities. We examine this problem through simulations because of the various advantages this approach offers, mainly efficiency, more control, and easy focus of the research. Furthermore, the simulation approach can be used because the simplifications that we made do not influence the value of the results. With this end, we have developed a framework to build multi-agent systems comprising affective agents and then, based on this platform, we developed an application for the exploration of unknown environments. This application is a simulated multi-agent environment in which, in addition to inanimate agents (objects), there are agents interacting in a simple way, whose goal is to explore the environment. By relying on an affective component plus ideas from the Belief-Desire-Intention model, our approach to building artificial agents is that of assigning agents mentalistic qualities such as feelings, basic desires, memory/beliefs, desires/goals, and intentions. The inclusion of affect in the agent architecture is supported by the psychological and neuroscience research over the past decades which suggests that emotions and, in general, motivations play a critical role in decision-making, action, and reasoning, by influencing a variety of cognitive processes (e.g., attention, perception, planning, etc.). Reflecting the primacy of those mentalistic qualities, the architecture of an agent includes the following modules: sensors, memory/beliefs (for entities - which comprises both analogical and propositional knowledge representations -, plans, and maps of the environment), desires/goals, intentions, basic desires (basic motivations/motives), feelings, and reasoning. The key components that determine the exhibition of the exploratory behaviour in an agent are the kind of basic desires, feelings, goals and plans with which the agent is equipped. Based on solid, psychological experimental evidence, an agent is equipped in advance with the basic desires for minimal hunger, maximal information gain (maximal reduction of curiosity), and maximal surprise, as well as with the correspondent feelings of hunger, curiosity and surprise. Each one of those basic desires drives the agent to reduce or to maximize a particular feeling. The desire for minimal hunger, maximal information gain and maximal surprise directs the agent, respectively, to reduce the feeling of hunger, to reduce the feeling of curiosity (by maximizing information gain) and to maximize the feeling of surprise. The desire to reduce curiosity does not mean that the agent dislike curiosity. Instead, it means the agent desires selecting actions whose execution maximizes the reduction of curiosity, i.e., actions that are preceded by maximal levels of curiosity and followed by minimal levels of curiosity, which corresponds to maximize information gain. The intensity of these feelings is, therefore, important to compute the degree of satisfaction of the basic desires. For the basic desires of minimal hunger and maximal surprise it is given by the expected intensities of the feelings of hunger and surprise, respectively, after performing an action, while for the desire of maximal information gain it is given by the intensity of the feeling of curiosity before performing the action (this is the expected information gain). The memory of an agent is setup with goals and decision-theoretic, hierarchical task-network plans for visiting entities that populate the environment, regions of the environment, and for going to places where the agent can recharge its battery. New goals are generated for each unvisited entity of the environment, for each place in the frontier of the explored area, and for recharging battery, by adapting past goals and plans to the current world state computed based on sensorial information and on the generation of expectations and assumptions for the gaps in the environment information provided by the sensors. These new goals and respective plans are then ranked according to their Expected Utility which reflects the positive and negative relevance for the basic desires of their accomplishment. The first one, i.e., the one with highest Expected Utility is taken as an intention. Besides evaluating the computational model of surprise, we experimentally investigated through simulations the following issues: the role of the exploration strategy (role of surprise, curiosity, and hunger), environment complexity, and amplitude of the visual field on the performance of the exploration of environments populated with entities; the role of the size or, to some extent, of the diversity of the memory of entities, and environment complexity on map-building by exploitation. The main results show that: the computational model of surprise is a satisfactory model of human surprise; the exploration of unknown environments populated with entities can be robustly and efficiently performed by affective agents (the strategies that rely on hunger combined or not with curiosity or surprise outperform significantly the others, being strong contenders to the classical strategy based on entropy and cost)