Mobile agent path planning under uncertain environment using reinforcement learning and probabilistic model checking

The major challenge in mobile agent path planning, within an uncertain environment, is effectively determining an optimal control model to discover the target location as quickly as possible and evaluating the control system's reliability. To address this challenge, we introduce a learning-verification integrated mobile agent path planning method to achieve both the effectiveness and the reliability. More specifically, we first propose a modified Q-learning algorithm (a popular reinforcement learning algorithm), called Q EA−learning algorithm, to find the best Q-table in the environment. We then determine the location transition probability matrix, and establish a probability model using the assumption that the agent selects a location with a higher Q-value. Secondly, the learnt behaviour of the mobile agent based on Q EA−learning algorithm, is formalized as a Discrete-time Markov Chain (DTMC) model. Thirdly, the required reliability requirements of the mobile agent control system are specified using Probabilistic Computation Tree Logic (PCTL). In addition, the DTMC model and the specified properties are taken as the input of the Probabilistic Model Checker PRISM for automatic verification. This is preformed to evaluate and verify the control system's reliability. Finally, a case study of a mobile agent walking in a grids map is used to illustrate the proposed learning algorithm. Here we have a special focus on the modelling approach demonstrating how PRISM can be used to analyse and evaluate the reliability of the mobile agent control system learnt via the proposed algorithm. The results show that the path identified using the proposed integrated method yields the largest expected reward.</p

Common Knowledge and Interactive Behaviors: A Survey

This paper surveys the notion of common knowledge taken from game theory and computer science. It studies and illustrates more generally the effects of interactive knowledge in economic and social problems. First of all, common knowledge is shown to be a central concept and often a necessary condition for coordination, equilibrium achievement, agreement, and consensus. We present how common knowledge can be practically generated, for example, by particular advertisements or leadership. Secondly, we prove that common knowledge can be harmful, essentially in various cooperation and negotiation problems, and more generally when there are con icts of interest. Finally, in some asymmetric relationships, common knowledge is shown to be preferable for some players, but not for all. The ambiguous welfare effects of higher-order knowledge on interactive behaviors leads us to analyze the role of decentralized communication in order to deal with dynamic or endogenous information structures.Interactive knowledge, common knowledge, information structure, communication.

Model Checking Trust-based Multi-Agent Systems

Trust has been the focus of many research projects, both theoretical and practical, in the recent years, particularly in domains where open multi-agent technologies are applied (e.g., Internet-based markets, Information retrieval, etc.). The importance of trust in such domains arises mainly because it provides a social control that regulates the relationships and interactions among agents. Despite the growing number of various multi-agent applications, they still encounter many challenges in their formal modeling and the verification of agents’ behaviors. Many formalisms and approaches that facilitate the specifications of trust in Multi-Agent Systems (MASs) can be found in the literature. However, most of these approaches focus on the cognitive side of trust where the trusting entity is normally capable of exhibiting properties about beliefs, desires, and intentions. Hence, the trust is considered as a belief of an agent (the truster) involving ability and willingness of the trustee to perform some actions for the truster. Nevertheless, in open MASs, entities can join and leave the interactions at any time. This means MASs will actually provide no guarantee about the behavior of their agents, which makes the capability of reasoning about trust and checking the existence of untrusted computations highly desired. This thesis aims to address the problem of modeling and verifying at design time trust in MASs by (1) considering a cognitive-independent view of trust where trust ingredients are seen from a non-epistemic angle, (2) introducing a logical language named Trust Computation Tree Logic (TCTL), which extends CTL with preconditional, conditional, and graded trust operators along with a set of reasoning postulates in order to explore its capabilities, (3) proposing a new accessibility relation which is needed to define the semantics of the trust modal operators. This accessibility relation is defined so that it captures the intuition of trust while being easily computable, (4) investigating the most intuitive and efficient algorithm for computing the trust set by developing, implementing, and experimenting different model checking techniques in order to compare between them in terms of memory consumption, efficiency, and scalability with regard to the number of considered agents, (5) evaluating the performance of the model checking techniques by analyzing the time and space complexity. The approach has been applied to different application domains to evaluate its computational performance and scalability. The obtained results reveal the effectiveness of the proposed approach, making it a promising methodology in practice

Multi-Valued Verification of Strategic Ability

Some multi-agent scenarios call for the possibility of evaluating specifications in a richer domain of truth values. Examples include runtime monitoring of a temporal property over a growing prefix of an infinite path, inconsistency analysis in distributed databases, and verification methods that use incomplete anytime algorithms, such as bounded model checking. In this paper, we present multi-valued alternating-time temporal logic (mv-ATL*), an expressive logic to specify strategic abilities in multi-agent systems. It is well known that, for branching-time logics, a general method for model-independent translation from multi-valued to two-valued model checking exists. We show that the method cannot be directly extended to mv-ATL*. We also propose two ways of overcoming the problem. Firstly, we identify constraints on formulas for which the model-independent translation can be suitably adapted. Secondly, we present a model-dependent reduction that can be applied to all formulas of mv-ATL*. We show that, in all cases, the complexity of verification increases only linearly when new truth values are added to the evaluation domain. We also consider several examples that show possible applications of mv-ATL* and motivate its use for model checking multi-agent systems

Epistemology of Intelligence Agencies

About the analogy between the epistemological and methodological aspects of the activity of intelligence agencies and some scientific disciplines, advocating for a more scientific approach to the process of collecting and analyzing information within the intelligence cycle. I assert that the theoretical, ontological and epistemological aspects of the activity of many intelligence agencies are underestimated, leading to incomplete understanding of current phenomena and confusion in inter-institutional collaboration. After a brief Introduction, which includes a history of the evolution of the intelligence concept after World War II, Intelligence Activity defines the objectives and organization of intelligence agencies, the core model of these organizations (the intelligence cycle), and the relevant aspects of the intelligence gathering and intelligence analysis. In the Ontology section, I highlight the ontological aspects and the entities that threaten and are threatened. The Epistemology section includes aspects specific to intelligence activity, with the analysis of the traditional (Singer) model, and a possible epistemological approach through the concept of tacit knowledge developed by scientist Michael Polanyi. In the Methodology section there are various methodological theories with an emphasis on structural analytical techniques, and some analogies with science, archeology, business and medicine. In Conclusions I argue on the possibility of a more scientific approach to methods of intelligence gathering and analysis of intelligence agencies. CONTENTS: Abstract 1 Introduction 1.1. History 2. Intelligence activity 2.1. Organizations 2.2. Intelligence cycle 2.3 Intelligence gathering 2.4. Intelligence analysis 2.5. Counterintelligence 2.6. Epistemic communities 3. Ontology 4. Epistemology 4.1. The tacit knowledge (Polanyi) 5. Methodologies 6. Analogies with other disciplines 6.1. Science 6.2. Archeology 6.3. Business 6.4. Medicine 7. Conclusions Bibliography DOI: 10.13140/RG.2.2.12971.4944

Formal methods for analysing, coordinating, and controlling decisions in multi-agent systems

Multiagentensysteme sind verteilte (Computer)Systeme, die sich aus autonomen interagierenden Systemkomponenten, bezeichnet als Agenten, zusammensetzen. Sie bieten ein flexibles Framework zur Modellierung und Analyse von interaktiven Systemen, in denen Kooperation, Eigeninteresse und Autonomie eine entscheidende Rolle spielen. Dies ist zum Beispiel der Fall in Smart Grids. Eine Herausforderung in solchen Systemen ist die Kontrolle und die Koordination von Systemausführungen. Agenten handeln autonom und lassen sich daher oftmals nicht direkt kontrollieren, sondern bestenfalls beeinflussen. Aufgrund der Autonomie und des Selbstinteresses, ist es schwierig, angemessene Kontrollmechanismen zu finden. Die vorliegende Arbeit behandelt formale Grundlagen zu den Themen Entscheidungsfindung, Koordination und Kontrolle in Multiagentensystemen. Insbesondere werden in diesem Zusammenhang Logiken zur Analyse und Spezifikation von strategischen Fähigkeiten von Agenten, unter diversen Restriktionen, untersucht. Es werden formale Ansätze zur Beeinflussung und Überwachung von Systemausführungen eingeführt. In einem weiteren Teil der Arbeit wird mittels spieltheoretischer Verfahren analysiert, wie rationale Agenten interagieren und Entscheidungen treffen. Es wird argumentiert, dass formale Methoden und Werkzeuge zur Analyse und Kontrolle von autonomen Systemen entscheidend für deren verlässliche Entwicklung sind.Multi-agent systems (MASs) are distributed (computer) systems composed of autonomously (inter-)acting system components referred to as agents. MASs offer a flexible framework to model and analyse many real world settings in which cooperation, self-interest, and autonomy are crucial elements. A key challenge in such settings is the control and coordination of behavior. However, due to the agents' autonomy behavior can often not be controlled, but at best be influenced in some way or another. For example, agents can be given incentives in order to affect their decision-making in such a way that the emergent behavior of all actors is desirable from the system's perspective. The properties of self-interest and autonomy make it challenging to find appropriate control mechanisms. Existing coordination and control approaches from the distributed system literature are often not applicable due to the lack of direct control on the system components of MASs. New methods and tools are needed. In this thesis formal foundations related to the subjects of decision making, coordination and control in MASs are studied. In particular, we investigate (extensions of) temporal and strategic logics which capture specific capabilities of agents that influence their decision making. We also propose formal approaches to control, coordinate and monitor the emergent behavior in MASs. In the last part of the thesis we analyse how rational agents interact and make decisions using game theoretical methods. We argue that such formal approaches and tools to analyse and control autonomous systems are crucial for the development of reliable and flexible systems and will become even more crucial in the near future

Seeing, Knowing, doing : case studies in modal logic

Dans le domaine des jeux vidéos par exemple, surtout des jeux de rôles, les personnages virtuels perçoivent un environnement, en tirent des connaissances puis effectuent des actions selon leur besoin. De même en robotique, un robot perçoit son environnement à l'aide de capteurs/caméras, établit une base de connaissances et effectuent des mouvements etc. La description des comportements de ces agents virtuels et leurs raisonnements peut s'effectuer à l'aide d'un langage logique. Dans cette thèse, on se propose de modéliser les trois aspects "voir", "savoir" et "faire" et leurs interactions à l'aide de la logique modale. Dans une première partie, on modélise des agents dans un espace géométrique puis on définit une relation épistémique qui tient compte des positions et du regard des agents. Dans une seconde partie, on revisite la logique des actions "STIT" (see-to-it-that ou "faire en sorte que") qui permet de faire la différence entre les principes "de re" et "de dicto", contrairement à d'autres logiques modales des actions. Dans une troisième partie, on s'intéresse à modéliser quelques aspects de la théorie des jeux dans une variante de la logique "STIT" ainsi que des émotions contre-factuelles comme le regret. Tout au long de cette thèse, on s'efforcera de s'intéresser aux aspects logiques comme les complétudes des axiomatisations et la complexité du problème de satisfiabilité d'une formule logique. L'intégration des trois concepts "voir", "savoir" et "faire" dans une et une seule logique est évoquée en conclusion et reste une question ouverte.Agents are entities who perceive their environment and who perform actions. For instance in role playing video games, ennemies are agents who perceive some part of the virtual world and who can attack or launch a sortilege. Another example may concern robot assistance for disabled people: the robot perceives obstacles of the world and can alert humans or help them. Here, we try to give formal tools to model knowledge reasoning about the perception of their environment and about actions based, on modal logic. First, we give combine the standard epistemic modal logic with perception constructions of the form (agent a sees agent b). We give a semantics in terms of position and orientation of the agents in the space that can be a line (Lineland) or a plane (Flatland). Concerning Lineland, we provide a complete axiomatization and an optimal procedure for model-checking and satisfiability problem. Concerning Flatland, we show that both model-checking and satisfiability problem are decidable but the exact complexities and the axiomatization remain open problems. Thus, the logics of Lineland and Flatland are completely a new approach: their syntax is epistemic but their semantics concern spatial reasoning. Secondly, we study on the logic of agency ``see-to-it-that'' STIT made up of construction of the form [J]A standing for ``the coalition of agents J sees to it that A''. Our interest is motivated: STIT is strictly more expressive that standard modal logic for agency like Coalition Logic CL or Alternating-time Temporal Logic ATL. In CL or ATL the ``de re'' and ``de dicto'' problem is quite difficult and technical whereas if we combine STIT-operators with epistemic operators, we can solve it in a natural way. However this strong expressivity has a prize: the general version of STIT is undecidable. That is why we focus on some syntactic fragments of STIT: either we restrict the allowed coalitions J in constructions [J]A or we restrict the nesting of modal STIT-operators. We provide axiomatizations and complexity results. Finally, we give flavour to epistemic modal logic by adding STIT-operators. The logic STIT is suitable to express counterfactual statements like ``agent a could have choosen an action such that A have been true''. Thus we show how to model counterfactual emotions like regret, rejoicing, disappointment and elation in this framework. We also model epistemic games by adapting the logic STIT by giving explicitely names of actions in the language. In this framework, we can model the notion of rational agents but other kind of behaviour like altruism etc., Nash equilibrium and iterated deletion of strictly dominated strategies