Argumentation accelerated reinforcement learning

Reinforcement Learning (RL) is a popular statistical Artificial Intelligence (AI) technique for building autonomous agents, but it suffers from the curse of dimensionality: the computational requirement for obtaining the optimal policies grows exponentially with the size of the state space. Integrating heuristics into RL has proven to be an effective approach to combat this curse, but deriving high-quality heuristics from people’s (typically conflicting) domain knowledge is challenging, yet it received little research attention. Argumentation theory is a logic-based AI technique well-known for its conflict resolution capability and intuitive appeal. In this thesis, we investigate the integration of argumentation frameworks into RL algorithms, so as to improve the convergence speed of RL algorithms. In particular, we propose a variant of Value-based Argumentation Framework (VAF) to represent domain knowledge and to derive heuristics from this knowledge. We prove that the heuristics derived from this framework can effectively instruct individual learning agents as well as multiple cooperative learning agents. In addition,we propose the Argumentation Accelerated RL (AARL) framework to integrate these heuristics into different RL algorithms via Potential Based Reward Shaping (PBRS) techniques: we use classical PBRS techniques for flat RL (e.g. SARSA(λ)) based AARL, and propose a novel PBRS technique for MAXQ-0, a hierarchical RL (HRL) algorithm, so as to implement HRL based AARL. We empirically test two AARL implementations — SARSA(λ)-based AARL and MAXQ-based AARL — in multiple application domains, including single-agent and multi-agent learning problems. Empirical results indicate that AARL can improve the convergence speed of RL, and can also be easily used by people that have little background in Argumentation and RL.Open Acces

Argumentation for machine learning: a survey

Existing approaches using argumentation to aid or improve machine learning differ in the type of machine learning technique they consider, in their use of argumentation and in their choice of argumentation framework and semantics. This paper presents a survey of this relatively young field highlighting, in particular, its achievements to date, the applications it has been used for as well as the benefits brought about by the use of argumentation, with an eye towards its future

TOWARDS A WITTGENSTEINEAN LADDER FOR THE UNIVERSAL VIRTUAL CLASSROOM (UVC)

The aim of this work is to move from the foreign dominated to the self-dominated by encouraging people to draw their own conclusions with the help of own rational consideration. Here a room as an environment that is encouraging innovation, which can be denoted as “Innovation Lab”, and making processes as can be regarded as “Smart Lab” is an essential base. The question related to this generalized self-organizational learning method investigated in our paper is how a UVC, which is a room that connects people from different physical places to one synchronous and virtual perceivable place, which is built on these preconditions, can be operated both resource and learning-efficient for both the course participants and the educational organization. A practical approach of implementing a virtual classroom concept, including informative tutorial-feedback, is developed conceptually that also accounts for and implements the results of reinforcement machine-learning methods in AI applications. The difference that makes the difference is gained by reimplementing the AI tools in an AI instrument, in a “Smart Lab” environment and that in the teaching environment. By means of this, a cascaded feedback-loop system is informally installed, which gains feedback at different levels of abstraction. By this learning on each stage, in a collaborative and together decentralized and sequential fashion takes place, as the selforganizational implementations lead implicitly, also by means of the in the course implemented tools, to increasingly self-control. As such in the course, a tool is implemented, as generalizations by means of reinforcement learnings are to be emergently foreseen by this method, which goes beyond the tools, that have already been implemented before. This AI-enhanced learning coevolution shall then, predictively, as well increase the potential of the course participants as the educational organization according to the Wittgensteinean parable: A ladder leading into a selfly-organized future

Explain what you see:argumentation-based learning and robotic vision

In this thesis, we have introduced new techniques for the problems of open-ended learning, online incremental learning, and explainable learning. These methods have applications in the classification of tabular data, 3D object category recognition, and 3D object parts segmentation. We have utilized argumentation theory and probability theory to develop these methods. The first proposed open-ended online incremental learning approach is Argumentation-Based online incremental Learning (ABL). ABL works with tabular data and can learn with a small number of learning instances using an abstract argumentation framework and bipolar argumentation framework. It has a higher learning speed than state-of-the-art online incremental techniques. However, it has high computational complexity. We have addressed this problem by introducing Accelerated Argumentation-Based Learning (AABL). AABL uses only an abstract argumentation framework and uses two strategies to accelerate the learning process and reduce the complexity. The second proposed open-ended online incremental learning approach is the Local Hierarchical Dirichlet Process (Local-HDP). Local-HDP aims at addressing two problems of open-ended category recognition of 3D objects and segmenting 3D object parts. We have utilized Local-HDP for the task of object part segmentation in combination with AABL to achieve an interpretable model to explain why a certain 3D object belongs to a certain category. The explanations of this model tell a user that a certain object has specific object parts that look like a set of the typical parts of certain categories. Moreover, integrating AABL and Local-HDP leads to a model that can handle a high degree of occlusion

Partial primary reinforcement as a parameter of secondary reinforcement

Thesis (Ph.D.)--Boston UniversityThe problem of this paper is to investigate partial primary reinforcement as a possible parameter of secondary reinforcement. Although partial primary reinforcement is known to be important in many learning situations, there appears to be little systematic knowledge of its relationship to secondary reinforcement. An experiment was performed in which (1) a neutral stimulus was present on every training trial, (2) a primary reinforcer was present on only some of these trials, (3) after training was completed, a test was made for the secondary reinforcing properties of the neutral stimulus. Six independent groups of albino rats were trained in a simple runway with food as the primary reinforcer and goal box brightness as the neutral stimulus. Each group received a different number of primary reinforcements, namely, 100%, 90%, 80%, 60%, 40%, and 20%, out of one-hundred-twenty training trials. Half of the subjects were trained on a white goal box and half on a black goal box. When training was completed, the alleyway was converted to a T maze with black and white goal boxes. Neither goal box was visible to the subjects until after entrance. The animals were given twenty trials in the T maze, and the number of times they entered each goal box was tabulated. Analysis of the data revealed that the lower the percentage of reinforcement given during training, the greater were the number of entries into the training box during the test. Some characteristics of the function were: between 100% and 90% the strength of secondary reinforcement did not increase, between 90% and 80% there was a large increase, from 80% to 40% there was a further increase, and from 40% to 20% there was some decrease. It was also revealed that some subjects in the lower percentage of reinforcement groups went either to the training box or to the novel box on every test trial. Other aspects of the data were also analyzed. From this data a number of conclusions were drawn: 1. Partial primary reinforcement is a parameter of secondary reinforcement. Decrease in partial reinforcement results in an increase in secondary reinforcement various characteristics of this relationship were discussed. It was pointed out that the obtained function might be derived from two separate functions: the relationship of secondary reinforcement to the number of reinforced trials, and the relationship of secondary reinforcement to the number of non-reinforced trials. 2. The fact that some subjects went to the same box on every test trial was explained in terms of the development of strong secondary reinforcement, in the case of subjects who went to the training box, and in terms of the development of strong generalized secondary reinforcement, in the case of subjects who went to the novel box. 3. It has often been reported in the experimental literature that partially reinforced subjects show greater resistance to extinction than continuously reinforced subjects. Our findings can be applied to this phenomenon. Stimuli present during partial reinforcement are apt to acquire greater secondary reinforcing properties than those present during continuous reinforcement, and, hence, the presence of the former during extinction are able to maintain a higher frequency of responding than the presence of the latter. This hypothesis was distinguished from others offered in the literature which purport to explain the greater resistance to extinction in terms of secondary reinforcement. 4. It was pointed out that this experiment revealed a significant variable, secondary reinforcement, which might develop in studies whose training set up resembles ours. 5. Minor findings of the experiment were discussed

MARLeME: A Multi-Agent Reinforcement Learning Model Extraction Library.

Multi-Agent Reinforcement Learning (MARL) en-compasses a powerful class of methodologies that have beenapplied in a wide range of fields. An effective way to furtherempower these methodologies is to develop approaches and toolsthat could expand their interpretability and explainability. Inthis work, we introduce MARLeME: a MARL model extractionlibrary, designed to improve explainability of MARL systemsby approximating them with symbolic models. Symbolic modelsoffer a high degree of interpretability, well-defined properties,and verifiable behaviour. Consequently, they can be used toinspect and better understand the underlying MARL systemsand corresponding MARL agents, as well as to replace all/someof the agents that are particularly safety and security critical.In this work, we demonstrate how MARLeME can be appliedto two well-known case studies (Cooperative Navigation andRoboCup Takeaway), using extracted models based on AbstractArgumentation

Writing for different disciplines

About the book: Student academic writing is at the heart of teaching and learning in higher education. Students are assessed largely by what they write, and need to learn both general academic conventions as well as disciplinary writing requirements in order to be successful in higher education. Teaching Academic Writing is a 'toolkit' designed to help higher education lecturers and tutors teach writing to their students. Containing a range of diverse teaching strategies, the book offers both practical activities to help students develop their writing abilities and guidelines to help lecturers and tutors think in more depth about the assessment tasks they set and the feedback they give to students. The authors explore a wide variety of text types, from essays and reflective diaries to research projects and laboratory reports. The book draws on recent research in the fields of academic literacy, second language learning, and linguistics. It is grounded in recent developments such as the increasing diversity of the student body, the use of the Internet, electronic tuition, and issues related to distance learning in an era of increasing globalisation

CRITICAL AND CREATIVE THINKING IN THE WRITING OF THE EXPOSITION TEXT

Critical and creative thinking is a high-level thinking skill that needs to be trained to students as a provision to be a successful learner. Creative-critical thinking skills can be trained through text writing. One of the relevant texts for developing students' critical-creative thinking skills is exposition texts. This is because the exposition texts contain the opinions of a person who needs to be criticized for his supporting arguments and necessary creative ideas in solving the problem