Machine Learning Approach for an Advanced Agent-based Intelligent Tutoring System

Machine Learning Approach for an Advanced Agent-based Intelligent Tutoring System Roya Aminikia Learning Management Systems (LMSs) are digital frameworks that provide curriculum, training materials, and corresponding assessments to guarantee an effective learning process. Although these systems are capable of distributing the learning content, they do not support dynamic learning processes and do not have the capability to communicate with human learners who are required to interact in a dynamic environment during the learning process. To create this process and support the interaction feature, LMSs are equipped with Intelligent Tutoring Systems (ITSs). The main objective of an ITS is to facilitate students’ movement towards their learning goals through virtual tutoring. When equipped with ITSs, LMSs operate as dynamic systems to provide students with access to a tutor who is available anytime during the learning session. The crucial issues we address in this thesis are how to set up a dynamic LMS, and how to design the logical structure behind an ITS. Artificial intelligence, multi-agent technology and machine learning provide powerful theories and foundations that we leverage to tackle these issues. We designed and implemented the new concept of Pedagogical Agent (PA) as the main part of our ITS. This agent uses an evaluation procedure to compare each particular student, in terms of performance, with their peers to develop a worthwhile guidance. The agent captures global knowledge of students’ feature measurements during students’ guiding process. Therefore, the PA retains an updated status, called image, of each specific student at any moment. The agent uses this image for the purpose of diagnosing students’ skills to implement a specific correct instruction. To develop the infrastructure of the agent decision making algorithm, we laid out a protocol (decision tree) to select the best individual direction. The significant capability of the agent is the ability to update its functionality by looking at a student’s image at run time. We also applied two supervised machine learning methods to improve the decision making protocol performance in order to maximize the effect of the collaborating mechanism between students and the ITS. Through these methods, we made the necessary modifications to the decision making structure to promote students’ performance by offering prompts during the learning sessions. The conducted experiments showed that the proposed system is able to efficiently classify students into learners with high versus low performance. Deployment of such a model enabled the PA to use different decision trees while interacting with students of different learning skills. The performance of the system has been shown by ROC curves and details regarding combination of different attributes used in the two machine learning algorithms are discussed, along with the correlation of key attributes that contribute to the accuracy and performance of the decision maker components

Multi-Agent Control in Sociotechnical Systems

Process control is essential in chemical engineering and has diverse applications in automation, manufacturing, scheduling, etc. In this cross-disciplinary work, we shift the domain focus from the control of machines to the control of multiple intelligent agents. Our goal is to improve the optimization problem-solving process, such as optimal regulation of emerging technologies, in a multi-agent system. Achieving that improvement would have potential value both within and outside the chemical engineering community. This work also illustrates the possibility of applying process systems engineering techniques, especially process control, beyond chemical plants. It is very common to observe crowds of individuals solving similar problems with similar information in a largely independent manner. We argue here that the crowds can become more efficient and robust problem-solvers, by partially following the average opinion. This observation runs counter to the widely accepted claim that the wisdom of crowds deteriorates with social influence. The key difference is that individuals are self-interested and hence will reject feedbacks that do not improve their performance. We propose a multi-agent control-theoretic methodology, soft regulation, to model the collective dynamics and compute the degree of social influence, i.e., the level to which one accepts the population feedback, that optimizes the problem-solving performance. Soft regulation is a modeling language for multi-agent sociotechnical systems. The state-space formulation captures the individual learning process (i.e., open loop dynamics) as well as the influence of the population feedback in a straightforward manner. It can model a diverse set of existing multi-agent dynamics. Through numerical analysis and linear algebra, we attempt to understand the role of feedback in multi-agent collective dynamics, thus achieving multi-agent control in sociotechnical systems. Our analysis through mathematical proofs, simulations, and a human subject experiment suggests that intelligent individuals, solving the same problem (or similar problems), could do much better by adaptively adjusting their decisions towards the population average. We even discover that the crowd of human subjects could self-organize into a near-optimal setting. This discovery suggests a new coordination mechanism for enhancing individual decision-making. Potential applications include mobile health, urban planning, and policymaking

Challenges for a CBR framework for argumentation in open MAS

[EN] Nowadays, Multi-Agent Systems (MAS) are broadening their applications to open environments, where heterogeneous agents could enter into the system, form agents’ organizations and interact. The high dynamism of open MAS gives rise to potential conflicts between agents and thus, to a need for a mechanism to reach agreements. Argumentation is a natural way of harmonizing conflicts of opinion that has been applied to many disciplines, such as Case-Based Reasoning (CBR) and MAS. Some approaches that apply CBR to manage argumentation in MAS have been proposed in the literature. These improve agents’ argumentation skills by allowing them to reason and learn from experiences. In this paper, we have reviewed these approaches and identified the current contributions of the CBR methodology in this area. As a result of this work, we have proposed several open issues that must be taken into consideration to develop a CBR framework that provides the agents of an open MAS with arguing and learning capabilities.This work was partially supported by CONSOLIDER-INGENIO 2010 under grant CSD2007-00022 and by the Spanish government and FEDER funds under TIN2006-14630-C0301 project.Heras Barberá, SM.; Botti Navarro, VJ.; Julian Inglada, VJ. (2009). Challenges for a CBR framework for argumentation in open MAS. Knowledge Engineering Review. 24(4):327-352. https://doi.org/10.1017/S0269888909990178S327352244Willmott S. , Vreeswijk G. , Chesñevar C. , South M. , McGinnis J. , Modgil S. , Rahwan I. , Reed C. , Simari G. 2006. Towards an argument interchange format for multi-agent systems. In Proceedings of the AAMAS International Workshop on Argumentation in Multi-Agent Systems, ArgMAS-06, 17–34.Sycara, K. P. (1990). Persuasive argumentation in negotiation. Theory and Decision, 28(3), 203-242. doi:10.1007/bf00162699Ontañón S. , Plaza E. 2006. Arguments and counterexamples in case-based joint deliberation. In AAMAS-06 Workshop on Argumentation in Multi-Agent Systems, ArgMAS-06, 36–53.Sadri F. , Toni F. , Torroni P. 2001. Dialogues for negotiation: agent varieties and dialogue sequences. In Proceedings of the 8th International Workshop on Agent Theories, Architectures, and Languages, ATAL-01, Intelligent Agents VIII 2333, 405–421. Springer.Fox J. , Parsons S. 1998. Arguing about beliefs and actions. In Applications of Uncertainty Formalisms, Lecture Notes in Computer Science 1455, 266–302. Springer.Dung, P. M. (1995). On the acceptability of arguments and its fundamental role in nonmonotonic reasoning, logic programming and n-person games. Artificial Intelligence, 77(2), 321-357. doi:10.1016/0004-3702(94)00041-xAulinas M. , Tolchinsky P. , Turon C. , Poch M. , Cortés U. 2007. Is my spill environmentally safe? Towards an integrated management of wastewater in a river basin using agents that can argue. In 7th International IWA Symposium on Systems Analysis and Integrated Assessment in Water Management. Washington DC, USA.Amgoud L. 2003. A formal framework for handling conflicting desires. In Symbolic and Quantitative Approaches to Reasoning with Uncertainty, Lecture Notes in Computer Science 2711, 552–563. Springer.Armengol E. , Plaza E. 2001. Lazy induction of descriptions for relational case-based learning. In European Conference on Machine Learning, ECML-01, 13–24.Sørmo, F., Cassens, J., & Aamodt, A. (2005). Explanation in Case-Based Reasoning–Perspectives and Goals. Artificial Intelligence Review, 24(2), 109-143. doi:10.1007/s10462-005-4607-7RAHWAN, I., RAMCHURN, S. D., JENNINGS, N. R., McBURNEY, P., PARSONS, S., & SONENBERG, L. (2003). Argumentation-based negotiation. The Knowledge Engineering Review, 18(4), 343-375. doi:10.1017/s0269888904000098Brüninghaus S. , Ashley K. D. 2001. Improving the representation of legal case texts with information extraction methods. In 7th International Conference on Artificial Intelligence and Law, ICAIL-01, 42–51.Parsons, S. (1998). Agents that reason and negotiate by arguing. Journal of Logic and Computation, 8(3), 261-292. doi:10.1093/logcom/8.3.261Atkinson, K., Bench-Capon, T., & Mcburney, P. (2005). A Dialogue Game Protocol for Multi-Agent Argument over Proposals for Action. Autonomous Agents and Multi-Agent Systems, 11(2), 153-171. doi:10.1007/s10458-005-1166-xBrüninghaus S. , Ashley K. D. 2003. Predicting the outcome of case-based legal arguments. In 9th International Conference on Artificial Intelligence and Law, ICAIL-03, 233–242.Modgil S. , Tolchinsky P. , Cortés U. 2005. Towards formalising agent argumentation over the viability of human organs for transplantation. In 4th Mexican International Conference on Artificial Intelligence, MICAI-05, 928–938.Tolchinsky P. , Atkinson K. , McBurney P. , Modgil S. , Cortés U. 2007. Agents deliberating over action proposals using the ProCLAIM model. In 5th International Central and Eastern European Conference on Multi-Agent Systems, CEEMAS-07, 32–41.Prakken, H., & Sartor, G. (1998). Artificial Intelligence and Law, 6(2/4), 231-287. doi:10.1023/a:1008278309945Gordon T. F. , Karacapilidis N. 1997. The Zeno argumentation framework. In International Conference on Artificial Intelligence and Law, ICAIL-97, ACM Press, 10–18.Tolchinsky P. , Modgil S. , Cortés U. 2006a. Argument schemes and critical questions for heterogeneous agents to argue over the viability of a human organ. In AAAI Spring Symposium Series; Argumentation for Consumers of Healthcare, 377–384.Aleven V. , Ashley K. D. 1997. Teaching case-based argumentation through a model and examples, empirical evaluation of an intelligent learning environment. In 8th World Conference of the Artificial Intelligence in Education Society, 87–94.Rahwan, I. (2005). Guest Editorial: Argumentation in Multi-Agent Systems. Autonomous Agents and Multi-Agent Systems, 11(2), 115-125. doi:10.1007/s10458-005-3079-0RISSLAND, E. L., ASHLEY, K. D., & BRANTING, L. K. (2005). Case-based reasoning and law. The Knowledge Engineering Review, 20(3), 293-298. doi:10.1017/s0269888906000701Tolchinsky, P., Cortes, U., Modgil, S., Caballero, F., & Lopez-Navidad, A. (2006). Increasing Human-Organ Transplant Availability: Argumentation-Based Agent Deliberation. IEEE Intelligent Systems, 21(6), 30-37. doi:10.1109/mis.2006.116McBurney, P., Hitchcock, D., & Parsons, S. (2006). The eightfold way of deliberation dialogue. International Journal of Intelligent Systems, 22(1), 95-132. doi:10.1002/int.20191Rissland, E. L., Ashley, K. D., & Loui, R. P. (2003). AI and Law: A fruitful synergy. Artificial Intelligence, 150(1-2), 1-15. doi:10.1016/s0004-3702(03)00122-xSoh, L.-K., & Tsatsoulis, C. (2005). A Real-Time Negotiation Model and A Multi-Agent Sensor Network Implementation. Autonomous Agents and Multi-Agent Systems, 11(3), 215-271. doi:10.1007/s10458-005-0539-5Capobianco, M., Chesñevar, C. I., & Simari, G. R. (2005). Argumentation and the Dynamics of Warranted Beliefs in Changing Environments. Autonomous Agents and Multi-Agent Systems, 11(2), 127-151. doi:10.1007/s10458-005-1354-8Tolchinsky P. , Modgil S. , Cortés U. , Sànchez-Marrè M. 2006b. CBR and argument schemes for collaborative decision making. In Conference on Computational Models of Argument, COMMA-06, 144, 71–82. IOS Press.Ossowski S. , Julian V. , Bajo J. , Billhardt H. , Botti V. , Corchado J. M. 2007. Open issues in open MAS: an abstract architecture proposal. In Conferencia de la Asociacion Española para la Inteligencia Artificial, CAEPIA-07, 2, 151–160.Karacapilidis, N., & Papadias, D. (2001). Computer supported argumentation and collaborative decision making: the HERMES system. Information Systems, 26(4), 259-277. doi:10.1016/s0306-4379(01)00020-5Aamodt A. 2004. Knowledge-intensive case-based reasoning in Creek. In 7th European Conference on Case-Based Reasoning ECCBR-04, 1–15.Jakobovits H. , Vermeir D. 1999. Dialectic semantics for argumentation frameworks. In Proceedings of the 7th International Conference on Artificial Intelligence and Law, ICAIL-99, ACM Press, 53–62.Díaz-Agudo, B., & González-Calero, P. A. (s. f.). An Ontological Approach to Develop Knowledge Intensive CBR Systems. Ontologies, 173-213. doi:10.1007/978-0-387-37022-4_7Reed C. , Walton D. 2005. Towards a formal and implemented model of argumentation schemes in agent communication. In Proceedings of the 1st International Workshop in Multi-Agent Systems, ArgMAS-04, 173–188.Sycara K. 1989. Argumentation: planning other agents’ plans. In 11th International Joint Conference on Artificial Intelligence, 1, 517–523. Morgan Kaufmann Publishers, Inc.Bench-Capon, T. J. M., & Dunne, P. E. (2007). Argumentation in artificial intelligence. Artificial Intelligence, 171(10-15), 619-641. doi:10.1016/j.artint.2007.05.001Reiter, R. (1980). A logic for default reasoning. Artificial Intelligence, 13(1-2), 81-132. doi:10.1016/0004-3702(80)90014-4Amgoud L. , Kaci S. 2004. On the generation of bipolar goals in argumentation-based negotiation. In 1st International Workshop on Argumentation in Multi-Agent Systems, ArgMAS, Lecture Notes in Computer Science 3366, 192–207. Springer.CHESÑEVAR, C., MCGINNIS, MODGIL, S., RAHWAN, I., REED, C., SIMARI, G., … WILLMOTT, S. (2006). Towards an argument interchange format. The Knowledge Engineering Review, 21(4), 293-316. doi:10.1017/s0269888906001044Rahwan I. , Amgoud L. 2006. An argumentation-based approach for practical reasoning. In Proceedings of the 5th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS-06, ACM Press, 347–354.Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4(2), 155-169. doi:10.1007/bf01405730Soh L.-K. , Tsatsoulis C. 2001b. Reflective negotiating agents for real-time multisensor target tracking. In International Joint Conference on Artificial Intelligence, IJCAI-01, 1121–1127.Eemeren, F. H. van, & Grootendorst, R. (1984). Speech Acts in Argumentative Discussions. doi:10.1515/9783110846089Rissland E. L. , Skalak D. B. , Friedman M. T. 1993. Bankxx: a program to generate argument through case-based search. In International Conference on Artificial Intelligence and Law, ICAIL-93, 117–124.Sycara K. 1987. Resolving Adversarial Conflicts: An Approach Integrating Case-Based and Analytic Methods, PhD thesis, School of Information and Computer Science. Georgia Institute of Technology.Ontañón S. , Plaza E. 2007. Learning and joint deliberation through argumentation in multi-agent systems. In International Conference on Autonomous Agents and Multiagent Systems, AAMAS-07, 971–978.Rissland, E. L., & Skalak, D. B. (1991). CABARET: rule interpretation in a hybrid architecture. International Journal of Man-Machine Studies, 34(6), 839-887. doi:10.1016/0020-7373(91)90013-wDaniels J. J. , Rissland E. L. 1997. Finding legally relevant passages in case opinions. In 6th International Conference on Artificial Intelligence and Law, ICAIL-97, 39–47.Brüninghaus S. , Ashley K. D. 2005. Generating legal arguments and predictions from case texts. In 10th International Conference on Artificial Intelligence and Law, ICAIL-05, 65–74.Simari G. R. , García A. J. , Capobianco M. 2004. Actions, planning and defeasible reasoning. In Proceedings of the 10th International Workshop on Non-monotonic Reasoning, NMR-04, 377–384.Soh L.-K. , Tsatsoulis C. 2001a. Agent-based argumentative negotiations with case-based reasoning. In AAAI Fall Symposium on Negotiation Methods for Autonomous Cooperative Systems, 16–25.Ashley, K. D. (1991). Reasoning with cases and hypotheticals in HYPO. International Journal of Man-Machine Studies, 34(6), 753-796. doi:10.1016/0020-7373(91)90011-uHulstijn J. , van der Torre L. 2004, Combining goal generation and planning in an argumentation framework. In Proceedings of the Workshop on Argument, Dialogue and Decision. International Workshop on Non-monotonic Reasoning, NMR-04, 212–218.Karacapilidis N. , Trousse B. , Papadias D. 1997. Using case-based reasoning for argumentation with multiple viewpoints. In 2nd International Conference on Case-Based Reasoning, ICCBR-97, 541–552.Branting, L. K. (1991). Building explanations from rules and structured cases. International Journal of Man-Machine Studies, 34(6), 797-837. doi:10.1016/0020-7373(91)90012-

Virtual Interactions With Real-agents For Sustainable Natural Resource Management

Common pool resource management systems are complex to manage due to the absence of a clear understanding of the effects of users’ behavioral characteristics. Non-cooperative decision making based on individual rationality (as opposed to group rationality) and a tendency to free ride due to lack of trust and information about other users’ behavior creates externalities and can lead to tragedy of the commons without intervention by a regulator. Nevertheless, even regulatory institutions often fail to sustain natural common pool resources in the absence of clear understanding of the responses of multiple heterogeneous decision makers to different regulation schemes. While modeling can help with our understanding of complex coupled human-natural systems, past research has not been able to realistically simulate these systems for two major limitations: 1) lack of computational capacity and proper mathematical models for solving distributed systems with self-optimizing agents; and 2) lack of enough information about users’ characteristics in common pool resource systems due to absence of reliable monitoring information. Recently, different studies have tried to address the first limitation by developing agent-based models, which can be appropriately handled with today’s computational capacity. While these models are more realistic than the social planner’s models which have been traditionally used in the field, they normally rely on different heuristics for characterizing users’ behavior and incorporating heterogeneity. This work is a step-forward in addressing the second limitation, suggesting an efficient method for collecting information on diverse behavioral characteristics of real agents for incorporation in distributed agent-based models. Gaming in interactive virtual environments is suggested as a reliable method for understanding different variables that promote sustainable resource use through observation of decision making and iii behavior of the resource system beneficiaries under various institutional frameworks and policies. A review of educational or serious games for environmental management was undertaken to determine an appropriate game for collecting information on real-agents and also to investigate the state of environmental management games and their potential as an educational tool. A web-based groundwater sharing simulation game—Irrigania—was selected to analyze the behavior of real agents under different common pool resource management institutions. Participants included graduate and undergraduate students from the University of Central Florida and Lund University. Information was collected on participants’ resource use, behavior and mindset under different institutional settings through observation and discussion with participants. Preliminary use of water resources gaming suggests communication, cooperation, information disclosure, trust, credibility and social learning between beneficiaries as factors promoting a shift towards sustainable resource use. Additionally, Irrigania was determined to be an effective tool for complementing traditional lecture-based teaching of complex concepts related to sustainable natural resource management. The different behavioral groups identified in the study can be used for improved simulation of multi-agent groundwater management systems

Multi-agent evolutionary systems for the generation of complex virtual worlds

Modern films, games and virtual reality applications are dependent on convincing computer graphics. Highly complex models are a requirement for the successful delivery of many scenes and environments. While workflows such as rendering, compositing and animation have been streamlined to accommodate increasing demands, modelling complex models is still a laborious task. This paper introduces the computational benefits of an Interactive Genetic Algorithm (IGA) to computer graphics modelling while compensating the effects of user fatigue, a common issue with Interactive Evolutionary Computation. An intelligent agent is used in conjunction with an IGA that offers the potential to reduce the effects of user fatigue by learning from the choices made by the human designer and directing the search accordingly. This workflow accelerates the layout and distribution of basic elements to form complex models. It captures the designer's intent through interaction, and encourages playful discovery