Socially Aware Motion Planning with Deep Reinforcement Learning

For robotic vehicles to navigate safely and efficiently in pedestrian-rich environments, it is important to model subtle human behaviors and navigation rules (e.g., passing on the right). However, while instinctive to humans, socially compliant navigation is still difficult to quantify due to the stochasticity in people's behaviors. Existing works are mostly focused on using feature-matching techniques to describe and imitate human paths, but often do not generalize well since the feature values can vary from person to person, and even run to run. This work notes that while it is challenging to directly specify the details of what to do (precise mechanisms of human navigation), it is straightforward to specify what not to do (violations of social norms). Specifically, using deep reinforcement learning, this work develops a time-efficient navigation policy that respects common social norms. The proposed method is shown to enable fully autonomous navigation of a robotic vehicle moving at human walking speed in an environment with many pedestrians.Comment: 8 page

Soft behaviour modelling of user communities

A soft modelling approach for describing behaviour in on-line user communities is introduced in this work. Behaviour models of individual users in dynamic virtual environments have been described in the literature in terms of timed transition automata; they have various drawbacks. Soft multi/agent behaviour automata are defined and proposed to describe multiple user behaviours and to recognise larger classes of user group histories, such as group histories which contain unexpected behaviours. The notion of deviation from the user community model allows defining a soft parsing process which assesses and evaluates the dynamic behaviour of a group of users interacting in virtual environments, such as e-learning and e-business platforms. The soft automaton model can describe virtually infinite sequences of actions due to multiple users and subject to temporal constraints. Soft measures assess a form of distance of observed behaviours by evaluating the amount of temporal deviation, additional or omitted actions contained in an observed history as well as actions performed by unexpected users. The proposed model allows the soft recognition of user group histories also when the observed actions only partially meet the given behaviour model constraints. This approach is more realistic for real-time user community support systems, concerning standard boolean model recognition, when more than one user model is potentially available, and the extent of deviation from community behaviour models can be used as a guide to generate the system support by anticipation, projection and other known techniques. Experiments based on logs from an e-learning platform and plan compilation of the soft multi-agent behaviour automaton show the expressiveness of the proposed model

A flexible coupling approach to multi-agent planning under incomplete information

The final publication is available at Springer via http://dx.doi.org/10.1007/s10115-012-0569-7Multi-agent planning (MAP) approaches are typically oriented at solving loosely coupled problems, being ineffective to deal with more complex, strongly related problems. In most cases, agents work under complete information, building complete knowledge bases. The present article introduces a general-purpose MAP framework designed to tackle problems of any coupling levels under incomplete information. Agents in our MAP model are partially unaware of the information managed by the rest of agents and share only the critical information that affects other agents, thus maintaining a distributed vision of the task. Agents solve MAP tasks through the adoption of an iterative refinement planning procedure that uses single-agent planning technology. In particular, agents will devise refinements through the partial-order planning paradigm, a flexible framework to build refinement plans leaving unsolved details that will be gradually completed by means of new refinements. Our proposal is supported with the implementation of a fully operative MAP system and we show various experiments when running our system over different types of MAP problems, from the most strongly related to the most loosely coupled.This work has been partly supported by the Spanish MICINN under projects Consolider Ingenio 2010 CSD2007-00022 and TIN2011-27652-C03-01, and the Valencian Prometeo project 2008/051.Torreño Lerma, A.; Onaindia De La Rivaherrera, E.; Sapena Vercher, O. (2014). A flexible coupling approach to multi-agent planning under incomplete information. Knowledge and Information Systems. 38:141-178. https://doi.org/10.1007/s10115-012-0569-7S14117838Argente E, Botti V, Carrascosa C, Giret A, Julian V, Rebollo M (2011) An abstract architecture for virtual organizations: the THOMAS approach. Knowl Inf Syst 29(2):379–403Barrett A, Weld DS (1994) Partial-order planning: evaluating possible efficiency gains. Artif Intell 67(1):71–112Belesiotis A, Rovatsos M, Rahwan I (2010) Agreeing on plans through iterated disputes. In: Proceedings of the 9th international conference on autonomous agents and multiagent systems. pp 765–772Bellifemine F, Poggi A, Rimassa G (2001) JADE: a FIPA2000 compliant agent development environment. In: Proceedings of the 5th international conference on autonomous agents (AAMAS). ACM, pp 216–217Blum A, Furst ML (1997) Fast planning through planning graph analysis. Artif Intell 90(1–2):281–300Boutilier C, Brafman R (2001) Partial-order planning with concurrent interacting actions. J Artif Intell Res 14(105):136Brafman R, Domshlak C (2008) From one to many: planning for loosely coupled multi-agent systems. In: Proceedings of the 18th international conference on automated planning and scheduling (ICAPS). pp 28–35Brenner M, Nebel B (2009) Continual planning and acting in dynamic multiagent environments. J Auton Agents Multiag Syst 19(3):297–331Coles A, Coles A, Fox M, Long D (2010) Forward-chaining partial-order planning. In: Proceedings of the 20th international conference on automated planning and scheduling (ICAPS). pp 42–49Coles A, Fox M, Long D, Smith A (2008) Teaching forward-chaining planning with JavaFF. In: Colloquium on AI education, 23rd AAAI conference on artificial intelligenceCox J, Durfee E, Bartold T (2005) A distributed framework for solving the multiagent plan coordination problem. In: Proceedings of the 4th international joint conference on autonomous agents and multiagent systems (AAMAS). ACM, pp 821–827de Weerdt M, Clement B (2009) Introduction to planning in multiagent systems. Multiag Grid Syst 5(4):345–355Decker K, Lesser VR (1992) Generalizing the partial global planning algorithm. Int J Coop Inf Syst 2(2):319–346desJardins M, Durfee E, Ortiz C, Wolverton M (1999) A survey of research in distributed continual planning. AI Mag 20(4):13–22Doshi P (2007) On the role of interactive epistemology in multiagent planning. In: Artificial intelligence and, pattern recognition. pp 208–213Dréo J, Savéant P, Schoenauer M, Vidal V (2011) Divide-and-evolve: the marriage of descartes and darwin. In: Proceedings of the 7th international planning competition (IPC). Freiburg, GermanyDurfee EH (2001) Distributed problem solving and planning. In: Multi-agents systems and applications: selected tutorial papers from the 9th ECCAI advanced course (ACAI) and agentLink’s third European agent systems summer school (EASSS), vol LNAI 2086. Springer, pp 118–149Durfee EH, Lesser V (1991) Partial global planning: a coordination framework for distributed hypothesis formation. IEEE Trans Syst Man Cybern Special Issue Distrib Sens Netw 21(5):1167–1183Ephrati E, Rosenschein JS (1996) Deriving consensus in multiagent systems. Artif Intell 87(1–2):21–74Fikes R, Nilsson N (1971) STRIPS: a new approach to the application of theorem proving to problem solving. Artif Intell 2(3):189–208Fogués R, Alberola J, Such J, Espinosa A, Garcia-Fornes A (2010) Towards dynamic agent interaction support in open multiagent systems. In: Proceedings of the 2010 conference on artificial intelligence research and development: proceedings of the 13th international conference of the Catalan association for artificial intelligence’. IOS Press, pp 89–98Gerevini A, Long D (2006) Preferences and soft constraints in PDDL3. In: ICAPS workshop on planning with preferences and soft constraints, vol 6. Citeseer, pp 46–53Ghallab M, Howe A, Knoblock C, McDermott D, Ram A, Veloso M, Weld D, Wilkins D (1998) PDDL-the Planning Domain Definition Language. In: AIPS-98 planning committeeGmytrasiewicz P, Doshi P (2005) A framework for sequential planning in multi-agent settings. J Artif Intell Res 24:49–79Haslum P, Jonsson P (1999) Some results on the complexity of planning with incomplete information. In: Proceedings of the 5th European conference on, planning (ECP). pp 308–318Helmert M (2006) The fast downward planning system. J Artif Intell Res 26(1):191–246Hoffmann J, Nebel B (2001) The FF planning system: fast planning generation through heuristic search. J Artif Intell Res 14:253–302Jonsson A, Rovatsos M (2011) Scaling up multiagent planning: a best-response approach. In: Proceedings of the 21st international conference on automated planning and scheduling (ICAPS). AAAI, pp 114–121Kambhampati S (1997) Refinement planning as a unifying framework for plan synthesis. AI Mag 18(2):67–97Kaminka GA, Pynadath DV, Tambe M (2002) Monitoring teams by overhearing: a multi-agent plan-recognition approach. J Artif Intell Res 17:83–135Kone M, Shimazu A, Nakajima T (2000) The state of the art in agent communication languages. Knowl Inf Syst 2(3):259–284Kovacs DL (2011) Complete BNF description of PDDL3.1. Technical reportKraus S (1997) Beliefs, time and incomplete information in multiple encounter negotiations among autonomous agents. Ann Math Artif Intell 20(1–4):111–159Kumar A, Zilberstein S, Toussaint M (2011) Scalable multiagent planning using probabilistic inference. In: Proceedings of the 22nd international joint conference on artificial intelligence (IJCAI)’. Barcelona, Spain, pp 2140–2146Kvarnström J. (2011) Planning for loosely coupled agents using partial order forward-chaining. In: Proceedings of the 21st international conference on automated planning and scheduling (ICAPS). AAAI, pp 138–145Lesser V, Decker K, Wagner T, Carver N, Garvey A, Horling B, Neiman D, Podorozhny R, Prasad M, Raja A et al (2004) Evolution of the GPGP/TAEMS domain-independent coordination framework. Auton Agents Multi Agent Syst 9(1):87–143Lipovetzky N, Geffner H (2011) Searching for plans with carefully designed probes. In: Proceedings of the 21th international conference on automated planning and scheduling (ICAPS)Micacchi C, Cohen R (2008) A framework for simulating real-time multi-agent systems. Knowl Inf Syst 17(2):135–166Nguyen N, Katarzyniak R (2009) Actions and social interactions in multi-agent systems. Knowl Inf Syst 18(2):133–136Nguyen X, Kambhampati S (2001) Reviving partial order planning. In: Proceedings of the 17th international joint conference on artificial intelligence (IJCAI). Morgan Kaufmann, pp 459–464Nissim R, Brafman R, Domshlak C (2010) A general, fully distributed multi-agent planning algorithm. In: Proceedings of the 9th international conference on autonomous agents and multiagent systems (AAMAS). pp 1323–1330Pajares S, Onaindia E (2012) Defeasible argumentation for multi-agent planning in ambient intelligence applications. In: Proceedings of the 11th international conference on autonomous agents and multiagent systems (AAMAS) pp 509–516Paolucci M, Shehory O, Sycara K, Kalp D, Pannu A (2000) A planning component for RETSINA agents. Intelligent Agents VI. Agent Theories Architectures, and Languages pp 147–161Parsons S, Sierra C, Jennings N (1998) Agents that reason and negotiate by arguing. J Logic Comput 8(3):261Penberthy J, Weld D (1992) UCPOP: a sound, complete, partial order planner for ADL. In: Proceedings of the 3rd international conference on principles of knowledge representation and reasoning (KR). Morgan Kaufmann, pp 103–114Richter S, Westphal M (2010) The LAMA planner: guiding cost-based anytime planning with landmarks. J Artif Intell Res 39(1):127–177Sycara K, Pannu A (1998) The RETSINA multiagent system (video session): towards integrating planning, execution and information gathering. In: Proceedings of the 2nd international conference on autonomous agents (Agents). ACM, pp 350–351Tambe M (1997) Towards flexible teamwork. J Artif Intell Res 7:83–124Tang Y, Norman T, Parsons S (2010) A model for integrating dialogue and the execution of joint plans. Argumentation in multi-agent systems, pp 60–78Tonino H, Bos A, de Weerdt M, Witteveen C (2002) Plan coordination by revision in collective agent based systems. Artif Intell 142(2):121–145Van Der Krogt R, De Weerdt M (2005), Plan repair as an extension of planning. In: Proceedings of the 15th international conference on automated planning and scheduling (ICAPS). pp 161–170Weld D (1994) An introduction to least commitment planning. AI Mag 15(4):27Weld D (1999) Recent advances in AI planning. AI Mag 20(2):93–123Wilkins D, Myers K (1998) A multiagent planning architecture. In: Proceedings of the 4th international conference on artificial intelligence planning systems (AIPS), pp 154–162Wu F, Zilberstein S, Chen X (2011) Online planning for multi-agent systems with bounded communication. Artif Intell 175(2):487–511Younes H, Simmons R (2003) VHPOP: versatile heuristic partial order planner. J Artif Intell Res 20: 405–430Zhang J, Nguyen X, Kowalczyk R (2007) Graph-based multi-agent replanning algorithm. In: Proceedings of the 6th conference on autonomous agents and multiagent systems (AAMAS

A survey of agent-oriented methodologies

This article introduces the current agent-oriented methodologies. It discusses what approaches have been followed (mainly extending existing object oriented and knowledge engineering methodologies), the suitability of these approaches for agent modelling, and some conclusions drawn from the survey

Cognitive modeling of social behaviors

To understand both individual cognition and collective activity, perhaps the greatest opportunity today is to integrate the cognitive modeling approach (which stresses how beliefs are formed and drive behavior) with social studies (which stress how relationships and informal practices drive behavior). The crucial insight is that norms are conceptualized in the individual mind as ways of carrying out activities. This requires for the psychologist a shift from only modeling goals and tasks —why people do what they do—to modeling behavioral patterns—what people do—as they are engaged in purposeful activities. Instead of a model that exclusively deduces actions from goals, behaviors are also, if not primarily, driven by broader patterns of chronological and located activities (akin to scripts). To illustrate these ideas, this article presents an extract from a Brahms simulation of the Flashline Mars Arctic Research Station (FMARS), in which a crew of six people are living and working for a week, physically simulating a Mars surface mission. The example focuses on the simulation of a planning meeting, showing how physiological constraints (e.g., hunger, fatigue), facilities (e.g., the habitat’s layout) and group decision making interact. Methods are described for constructing such a model of practice, from video and first-hand observation, and how this modeling approach changes how one relates goals, knowledge, and cognitive architecture. The resulting simulation model is a powerful complement to task analysis and knowledge-based simulations of reasoning, with many practical applications for work system design, operations management, and training