Rule Value Reinforcement Learning for Cognitive Agents

RVRL (Rule Value Reinforcement Learning) is a new algorithm which extends an existing learning framework that models the environment of a situated agent using a probabilistic rule representation. The algorithm attaches values to learned rules by adapting reinforcement learning. Structure captured by the rules is used to form a policy. The resulting rule values represent the utility of taking an action if the rule`s conditions are present in the agent`s current percept. Advantages of the new framework are demonstrated, through examples in a predator-prey environment

Modelling Emotion Based Reward Valuation with Computational Reinforcement Learning

We show that computational reinforcement learning can model human decision making in the Iowa Gambling Task (IGT). The IGT is a card game, which tests decision making under uncertainty. In our experiments, we found that modulating learning rate decay in Q-learning, enables the approximation of both the behaviour of normal subjects and those who are emotionally impaired by ventromedial prefrontal lesions. Outcomes observed in impaired subjects are modeled by high learning rate decay, while low learning rate decay replicates healthy subjects under otherwise identical conditions. The ventromedial prefrontal cortex has been associated with emotion based reward valuation, and, the value function in reinforcement learning provides an analogous assessment mechanism. Thus reinforcement learning can provide a good model for the role of emotional reward as a modulator of the learning rate

Electrodes for sealed secondary batteries

Self-supporting membrane electrode structures, in which active ingredients and graphite are incorporated in a polymeric matrix, improve performance of electrodes in miniature, sealed, alkaline storage batteries

Learning to Act with RVRL Agents

The use of reinforcement learning to guide action selection of cognitive agents has been shown to be a powerful technique for stochastic environments. Standard Reinforcement learning techniques used to provide decision theoretic policies rely, however, on explicit state-based computations of value for each state-action pair. This requires the computation of a number of values exponential to the number of state variables and actions in the system. This research extends existing work with an acquired probabilistic rule representation of an agent environment by developing an algorithm to apply reinforcement learning to values attached to the rules themselves. Structure captured by the rules is then used to learn a policy directly. The resulting value attached to each rule represents the utility of taking an action if the conditions of the rule are present in the agent’s current set of percepts. This has several advantages for planning purposes: generalization over many states and over unseen states; effective decisions can therefore be made with less training data than state based modelling systems (e.g. Dyna Q-Learning); and the problem of computation in an exponential state-action space is alleviated. The results of application of this algorithm to rules in a specific environment are presented, with comparison to standard reinforcement learning policies developed from related work

NPCs as People, Too: The Extreme AI Personality Engine

PK Dick once asked “Do Androids Dream of Electric Sheep?” In video games, a similar question could be asked of non-player characters: Do NPCs have dreams? Can they live and change as humans do? Can NPCs have personalities, and can these develop through interactions with players, other NPCs, and the world around them? Despite advances in personality AI for games, most NPCs are still undeveloped and undeveloping, reacting with flat affect and predictable routines that make them far less than human— in fact, they become little more than bits of the scenery that give out parcels of information. This need not be the case. Extreme AI, a psychology-based personality engine, creates adaptive NPC personalities. Originally developed as part of the thesis “NPCs as People: Using Databases and Behaviour Trees to Give Non-Player Characters Personality,” Extreme AI is now a fully functioning personality engine using all thirty facets of the Five Factor model of personality and an AI system that is live throughout gameplay. This paper discusses the research leading to Extreme AI; develops the ideas found in that thesis; discusses the development of other personality engines; and provides examples of Extreme AI’s use in two game demos

Labour process theory and critical management studies

Labour Process Theory (LPT) is conventionally and rightly listed as one of the analytical resources for Critical Management Studies (CMS). Yet, the relationships between the two have been, in the words of a classic of the former, a contested terrain. This is hardly surprising. Even if we set aside the inevitable multiplicity of perspectives, there is a tension in potential objects of analysis. Before CMS burst on to the scene, LPT was being criticised at its peak of influence in the 1980s for paying too much attention to management and too little to capital(ism) and labour. This was sometimes attributed to the location of many of the protagonists (in the UK at least) in business schools, but was, more likely a reflection of wider theoretical and ideological divides

A Life of Scholarship and Service to the Communication Discipline: Celebrating Lawrence W. Hugenberg

A tribute to the Basic Communication Course Annual\u27s founding editor, Lawrence W. Hugenberg, who died on August 11, 2008

Development of a Virtual Laparoscopic Trainer using Accelerometer Augmented Tools to Assess Performance in Surgical training

Previous research suggests that virtual reality (VR) may supplement conventional training in laparoscopy. It may prove useful in the selection of surgical trainees in terms of their dexterity and spatial awareness skills in the near future. Current VR training solutions provide levels of realism and in some instances, haptic feedback, but they are cumbersome by being tethered and not ergonomically close to the actual surgical instruments for weight and freedom of use factors. In addition, they are expensive hence making them less accessible to departments than conventional box trainers. The box trainers in comparison, although more economical, lack tangible feedback and realism for handling delicate tissue structures. We have previously reported on the development of a modified digitally enhanced surgical instrument for laparoscopic training, named the Parkar Tool. This tool contains wireless accelerometer and gyroscopic sensors integrated into actual laparoscopic instruments. By design, it alleviates the need for both tethered and physically different shaped tools thereby enhancing the realism when performing surgical procedures. Additionally the software (Valhalla) has the ability to digitally record surgical motions, thereby enabling it to remotely capture surgical training data to analyse and objectively evaluate performance. We have adapted and further developed our initial single training tool method as used with a laparoscopic pyloromyotomy scenario, to an enhanced method using multiple Parkar wireless tools simultaneously, for use in several different case scenarios. This allows the use and measurement of right and left handed dexterity with the benefit of using several tasks of differing complexity. The development of a 3D tissue-surface deformations solution written in OpenGL gives us several different virtual surgical training scenario approximations to use with the instruments. The trainee can start with learning simple tasks e.g. incising tissue, grasping, squeezing and stretching tissue, to more complex procedures such as suturing, herniotomies, bowel anastomoses, as well as the original pyloromyotomy as used in the first model

The Marine Mammal Protection Act and the Fishery Conservation and Management Act: The Need for Balance

This article presents an analysis of those provisions of the MMPA which may impede the achievement of FCMA objectives. It is important that these possible conflicts be resolved because while the United States controls off the coast of Alaska what may be the world\u27s largest resources of fish, these same waters contain enormous numbers of marine mammals. These fishery resources, if managed rationally, can make a large contribution to the economy of the United States and to the protein needs of the world. A reasonable accommodation between the MMPA and the FCMA must be found in order to achieve that possibility

Conceptualization and operationalization: utility of communication privacy management theory

Communication Privacy Management (CPM) theory explains one of the most important, yet challenging social processes in everyday life, that is, managing disclosing and protecting private information. The CPM privacy management system offers researchers, students, and the public a comprehensive approach to the complex and fluid character of privacy management in action. Following an overview of Communication Privacy Management framework, this review focuses on recent research utilizing CPM concepts that cross a growing number of contexts and illustrates the way people navigate privacy in action. Researchers operationalize the use of privacy rules and other core concepts that help describe and explain the ups and downs of privacy management people encounter