Emele, Chukwuemeka D

Guerin, Frank

Norman, Timothy J

Parsons, Simon

English

We present an efficient approach for identifying, learning and modeling the policies of others during collaborative ac-tivities. In a set of experiments, we demonstrate that more accurate models of others ’ policies (or norms) can be de-veloped more rapidly using various forms of evidence from argumentation-based dialogue. Categories and Subject Descriptors I.2.11 [Distributed Artificial Intelligence]: Multiagen

Chukwuemeka D. Emele

Timothy J. Norman

Frank Guerin

Simon Parsons

CiteSeerX

S.: Learning policies through argumentation-derived evidence (extended abstract

Aberdeen University Research

Learning Policies through Argumentation-derivedEvidence(Extended Abstract)Chukwuemeka D. EmeleUniversity of Aberdeen,Aberdeen, AB24 3UE, UKc.emele@abdn.ac.ukTimothy J. NormanUniversity of Aberdeen,Aberdeen, AB24 3UE, UKt.j.norman@abdn.ac.ukFrank GuerinUniversity of Aberdeen,Aberdeen, AB24 3UE, UKf.guerin@abdn.ac.ukSimon ParsonsBrooklyn College, CityUniversity of New York, USAparsons@sci.brooklyn.cuny.eduABSTRACTWe present an efficient approach for identifying, learningand modeling the policies of others during collaborative ac-tivities. In a set of experiments, we demonstrate that moreaccurate models of others’ policies (or norms) can be de-veloped more rapidly using various forms of evidence fromargumentation-based dialogue.Categories and Subject DescriptorsI.2.11 [Distributed Artificial Intelligence]: MultiagentSystemsGeneral TermsAlgorithms, ExperimentationKeywordsArgumentation, Machine learning, Policies, Norms1. INTRODUCTIONMany distributed problem solving scenarios require theformation of a team of collaborating agents. In such scenar-ios agents often operate under constraints, which determineto a large extent, how they go about their assigned tasks.When these constraints are part of the standard operatingprocedures of the agents or the organisations they represent,we refer to them as policies (also known as norms). Al-though agents may have prior assumptions about the policiesthat constrain the activities of others, these models are oftenincomplete and may be inaccurate. In a set of experiments,we investigate how information acquired through dialogue,which we call argumentation-derived evidence (ADE), canbe effectively exploited to learn better models of agents’ poli-cies. Our claim is that, through the use of argumentation-Cite as: Learning Policies through Argumentation-derived Evidence(Extended Abstract), Chukwuemeka D. Emele, Timothy J. Norman, FrankGuerin and Simon Parsons,Proc. of 9th Int. Conf. on AutonomousAgents and Multiagent Systems (AAMAS 2010), van der Hoek,Kaminka, Lespérance, Luck and Sen (eds.), May, 10–14, 2010, Toronto,Canada, pp. XXX-XXX.Copyright c© 2010, International Foundation for Autonomous Agents andMultiagent Systems (www.ifaamas.org). All rights reserved.derived evidence, better (more complete and correct) mod-els of the policies of others can be learned. We demonstratethe validity of this claim through experiments conducted ina resource provisioning scenario [1]. The research presentedin this extended abstract represents the first model for usingevidence derived from argumentation to learn underlying so-cial characteristics (e.g. policies/norms) of others. There is,however, some prior research in combining machine learn-ing and argumentation, and in using argument structuresfor machine learning. Rovatsos et al. [4] use hierarchical re-inforcement learning in modifying symbolic constructs thatregulate agent conversation patterns, and argue that theirapproach could improve an agent’s conversation strategy. Inother research, Mozˇina et al. [2] propose a novel induction-based machine learning mechanism using argumentation. Inour work, we used information-seeking dialogue [6] to ob-tain evidence from the interaction and learned the entiresequence as against a segment (frame) of the interaction [4].2. EXPERIMENTSThe scenario adopted in this research involves agents act-ing as resource seekers, that interact with potential providersto resource a plan. In order to gather useful evidence, agentsengage in argumentative dialogue and attempt to learn fromdialogical encounters with other agents. The enactment ofboth seeker and provider roles are governed by individualpolicies that regulate their actions. A seeker agent requiresresources in order to carry out some assigned tasks. Theseeker agent generates requests in accordance with its poli-cies and negotiates with provider agents based on these con-straints. On the other hand, provider agents have accessto certain resources and may have policies that govern theprovision of such resources to other members of the team.Provider agents do not have an unlimited pool of resourcesand so some resources may be temporarily unavailable. Bya resource being available we mean that it is not committedto another task (or agent) at the time requested.In this study we used three different machine learningmechanisms: Decision tree learning, Instance-based learningand Rule-based learning. These three mechanisms representvery different classes of machine learning algorithms. Therationale for exploring a range of learning techniques is toTable 1: Average percentage of policies classified correctly and standard deviationTasks C4.5 - ADE C4.5 + ADE k-NN - ADE k-NN + ADE Prism - ADE Prism + ADE1000 58.3 ± 15.1 60.3 ± 14.4 65.2 ± 9.8 71.1 ± 9.0 66.7 ± 8.2 67.7 ± 7.72000 69.2 ± 16.6 75.0 ± 12.6 71.0 ± 7.8 85.9 ± 7.3 71.7 ± 6.0 87.1 ± 6.43000 75.1 ± 12.0 83.6 ± 6.5 75.3 ± 5.3 92.0 ± 4.6 78.7 ± 8.4 94.1 ± 4.24000 82.1 ± 12.3 89.9 ± 5.2 80.7 ± 3.8 96.8 ± 3.1 84.3 ± 6.5 96.6 ± 4.15000 85.3 ± 8.9 93.0 ± 3.4 81.0 ± 4.1 97.3 ± 3.6 87.4 ± 6.0 97.5 ± 2.66000 88.2 ± 8.2 95.6 ± 5.1 82.0 ± 3.8 98.4 ± 1.7 90.6 ± 5.3 99.2 ± 1.0demonstrate the utility of argumentation-derived evidenceregardless of the machine learning technique employed. Thus,our hypothesis is that the use of evidence acquired throughargumentation significantly improves the performance of ma-chine learning in the development and refinement of modelsof other agents. Decision tree learning (C4.5): In this ap-proach, the policies are conceived as concepts of an agent.Agent policies are represented as a vector of attributes andthese attributes are communicated back and forth during ne-gotiation. The C4.5 algorithm is then used to classify eachset of attributes (policy instance) into a class (that is, grantor deny). In future encounters, the seeker agent attempts topredict the policies of the provider based on the model it hasbuilt. 2. Instance-based learning (k-Nearest Neighbours): Inthis approach, policies are stored as instances and new policyinstances are classified based on the closest training exam-ples in the feature space. 3. Rule-based learning (Prism):This approach encode policies as rules, and use sequentialcovering to induce the rules by selecting attribute-value pairsthat satisfy the rule- one rule at a time. The policies coveredby the rule are then removed and the process iterates untilall the rules are covered.3. RESULTSExperiments were conducted with seeker agents initialisedwith random models of the policies of providers. 100 runswere conducted for each case, and tasks were randomly cre-ated during each run from 375 possible configurations.Table 1 illustrates the effectiveness of identifying and learn-ing policies through argumentation-derived evidence usingthe three machine learning techniques described above. Itshows the percentage of policies classified correctly and theirstandard deviations. In each case the model of others’ poli-cies is recomputed after each set of 1000 tasks. For allthree machine learning techniques considered, the percent-age of policies predicted correctly as a result of exploitingevidence derived from argumentation was consistently andsignificantly higher than those predicted without such evi-dence. This result shows that the exchange of argumentsduring practical dialogue enabled agents to learn and buildmore accurate models of other agents’ policies much fasterthan scenarios where there was no exchange of arguments.Tests of statistical significance were applied to the results.Using linear regression, the analysis of variance (ANOVA)shows that as the number of tasks increases, each of the threelearning techniques (with or without argumentation-derivedevidence) consistently converges with a 95% confidence in-terval. Furthermore, for all the pairwise comparisons, thescenarios where argumentation-derived evidence was com-bined with machine learning techniques consistently yieldedhigher rates of convergence (p < 0.02) than those withoutadditional evidence. These results confirm our hypothesis.4. CONCLUSIONSThe results of these experiments have shown that evidencederived from argumentation can have a statistically signifi-cant positive impact on identifying, learning and modelingothers’ policies during collaborative activities. Accurate pol-icy models can inform strategies for advising human decisionmakers on how a plan may be resourced and who to talk to[5], and may aid in the development of more effective strate-gies for agents [3]. Our results demonstrate that significantimprovements can be achieved by combining machine learn-ing techniques with argumentation-derived evidence.AcknowledgementsThis research was sponsored by the U.S. Army ResearchLaboratory and the U.K. Ministry of Defence and was ac-complished under Agreement Number W911NF-06-3-0001.The views and conclusions contained in this document arethose of the author(s) and should not be interpreted as rep-resenting the official policies, either expressed or implied, ofthe U.S. Army Research Laboratory, the U.S. Government,the U.K. Ministry of Defence or the U.K. Government. TheU.S. and U.K. Governments are authorized to reproduce anddistribute reprints for Government purposes notwithstand-ing any copyright notation hereon.5. REFERENCES[1] C. D. Emele, T. J. Norman, F. Guerin, and S. Parsons.Learning policy constraints through dialogue. In Proc.of the AAAI Fall Symposium on The Uses ofComputational Argumentation, pages 20–26, Virginia,USA, 2009.[2] M. Mozˇina, J. Zˇabkar, and I. Bratko. Argument basedmachine learning. Artificial Intelligence,171(10-15):922–937, 2007.[3] N. Oren, T. J. Norman, and A. Preece. Loose lips sinkships: A heuristic for argumentation. In Proc. of the3rd Int’l Workshop on Argumentation in Multi-AgentSystems (ArgMAS 2006), pages 121–134, 2006.[4] M. Rovatsos, I. Rahwan, F. Fischer, and G. Weiss.Practical strategic reasoning and adaptation in rationalargument-based negotiation. In Argumentation inMulti-Agent Systems, volume 4049 of LNCS, pages122–137. Springer-Berlin, 2005.[5] K. Sycara, T. J. Norman, J. A. Giampapa, M. J.Kollingbaum, C. Burnett, D. Masato, M. McCallum,and M. H. Strub. Agent support for policy-drivencollaborative mission planning. The Computer Journal,page bxp061, 2009.[6] D. N. Walton and E. C. W. Krabbe. Commitment inDialogue: Basic Concepts of Interpersonal Reasoning.SUNY Press, Albany, NY, USA, 1995.

Agent support for policy-driven collaborative mission planning. The Computer Journal,

Argument based machine learning.

Commitment in Dialogue: Basic Concepts of Interpersonal Reasoning.

Learning policy constraints through dialogue.

Loose lips sink ships: A heuristic for argumentation.

Practical strategic reasoning and adaptation in rational argument-based negotiation.

Learning policies through argumentation-derived evidence (extended abstract)

https://aura.abdn.ac.uk/bitstream/2164/758/1/AAMAS2010_0504_40e040e8c.pdf

Learning policies through argumentation-derived evidence (extended abstract)

Abstract

Similar works

Full text

Available Versions

CiteSeerX

Aberdeen University Research