Learning automata and its application to priority assignment in a queuing system with unknown characteristics /

Conditions for (epsilon)-optimality of a general class of absorbing barrier and strongly absolutely expedient learning algorithms are derived. As a consequence, a new class of learning algorithms having identical behavior under the occurrence of success and failure are obtained. An application of learning automata to the priority assignment in a queuing system with unknown characteristics is given

On the Theory and Applications of Hierarchical Learning Automata and Object Migration Automata

Paper III, IV and VIII are excluded due to copyright.The paradigm of Artificial Intelligence (AI) and the sub-group of Machine Learning (ML) have attracted exponential interest in our society in recent years. The domain of ML contains numerous methods, and it is desirable (and in one sense, mandatory) that these methods are applicable and valuable to real-life challenges. Learning Automata (LA) is an intriguing and classical direction within ML. In LA, non-human agents can find optimal solutions to various problems through the concept of learning. The LA instances learn through Agent-Environment interactions, where advantageous behavior is rewarded, and disadvantageous behavior is penalized. Consequently, the agent eventually, and hopefully, learns the optimal action from a set of actions. LA has served as a foundation for Reinforcement Learning (RL), and the field of LA has been studied for decades. However, many improvements can still be made to render these algorithms to be even more convenient and effective. In this dissertation, we record our research contributions to the design and applications within the field of LA. Our research includes novel improvements to the domain of hierarchical LA, major advancements to the family of Object Migration Automata (OMA) algorithms, and a novel application of LA, where it was utilized to solve challenges in a mobile radio communication system. More specifically, we introduced the novel Hierarchical Discrete Pursuit Automaton (HDPA), which significantly improved the state of the art in terms of effectiveness for problems with high accuracy requirements, and we mathematically proved its ϵ-optimality. In addition, we proposed the Action Distribution Enhanced (ADE) approach to hierarchical LA schemes which significantly reduced the number of iterations required before the machines reached convergence. The existing schemes in the OMA paradigm, which are able to solve partitioning problems, could only solve problems with equally-sized partitions. Therefore, we proposed two novel methods that could handle unequally-sized partitions. In addition, we rigorously summarized the OMA domain, outlined its potential benefits to society, and listed further development cases for future researchers in the field. With regard to applications, we proposed an OMA-based approach to the grouping and power allocation in Non-orthogonal Multiple Access (NOMA) systems, demonstrating the applicability of the OMA and its advantage in solving fairly complicated stochastic problems. The details of these contributions and their published scientific impacts will be summarized in this dissertation, before we present some of the research contributions in their entirety.publishedVersio

Reinforcement Learning

Brains rule the world, and brain-like computation is increasingly used in computers and electronic devices. Brain-like computation is about processing and interpreting data or directly putting forward and performing actions. Learning is a very important aspect. This book is on reinforcement learning which involves performing actions to achieve a goal. The first 11 chapters of this book describe and extend the scope of reinforcement learning. The remaining 11 chapters show that there is already wide usage in numerous fields. Reinforcement learning can tackle control tasks that are too complex for traditional, hand-designed, non-learning controllers. As learning computers can deal with technical complexities, the tasks of human operators remain to specify goals on increasingly higher levels. This book shows that reinforcement learning is a very dynamic area in terms of theory and applications and it shall stimulate and encourage new research in this field

Considerations in designing a cybernetic simple 'learning' model; and an overview of the problem of modelling learning

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Learning is viewed as a central feature of living systems and must be manifested in any artifact that claims to exhibit general intelligence. The central aims of the thesis are twofold: (1) - To review and critically assess the empirical and theoretical aspects of learning as have been addressed in a multitude of disciplines, with the aim of extracting fundamental features and elements. (2) - To develop a more systematic approach to the cybernetic modelling of learning than has been achieved hitherto. In pursuit of aim (1) above the following discussions are included: Historical and Philosophical backgrounds; Natural learning, both physiological and psychological aspects; Hierarchies of learning identified in the evolutionary, functional and developmental senses; An extensive section on the general problem of modelling of learning and the formal tools, is included as a link between aims (1) and (2). Following this a systematic and historically oriented study of cybernetic and other related approaches to the problem of modelling of learning is presented. This then leads to the development of a state-of-the-art general purpose experimental cybernetic learning model. The programming and use of this model is also fully described, including an elaborate scheme for the manifestation of simple learning

Algorithmic Compositional Methods and their Role in Genesis: A Multi-Functional Real-Time Computer Music System

Algorithmic procedures have been applied in computer music systems to generate compositional products using conventional musical formalism, extensions of such musical formalism and extra-musical disciplines such as mathematical models. This research investigates the applicability of such algorithmic methodologies for real-time musical composition, culminating in Genesis, a multi-functional real-time computer music system written for Mac OS X in the SuperCollider object-oriented programming language, and contained in the accompanying DVD. Through an extensive graphical user interface, Genesis offers musicians the opportunity to explore the application of the sonic features of real-time sound-objects to designated generative processes via different models of interaction such as unsupervised musical composition by Genesis and networked control of external Genesis instances. As a result of the applied interactive, generative and analytical methods, Genesis forms a unique compositional process, with a compositional product that reflects the character of its interactions between the sonic features of real-time sound-objects and its selected algorithmic procedures. Within this thesis, the technologies involved in algorithmic methodologies used for compositional processes, and the concepts that define their constructs are described, with consequent detailing of their selection and application in Genesis, with audio examples of algorithmic compositional methods demonstrated on the accompanying DVD. To demonstrate the real-time compositional abilities of Genesis, free explorations with instrumentalists, along with studio recordings of the compositional processes available in Genesis are presented in audiovisual examples contained in the accompanying DVD. The evaluation of the Genesis system’s capability to form a real-time compositional process, thereby maintaining real-time interaction between the sonic features of real-time sound objects and its selected algorithmic compositional methods, focuses on existing evaluation techniques founded in HCI and the qualitative issues such evaluation methods present. In terms of the compositional products generated by Genesis, the challenges in quantifying and qualifying its compositional outputs are identified, demonstrating the intricacies of assessing generative methods of compositional processes, and their impact on a resulting compositional product. The thesis concludes by considering further advances and applications of Genesis, and inviting further dissemination of the Genesis system and promotion of research into evaluative methods of generative techniques, with the hope that this may provide additional insight into the relative success of products generated by real-time algorithmic compositional processes

Computer Architecture in Industrial, Biomechanical and Biomedical Engineering

This book aims to provide state-of-the-art information on computer architecture and simulation in industry, engineering, and clinical scenarios. Accepted submissions are high in scientific value and provide a significant contribution to computer architecture. Each submission expands upon novel and innovative research where the methods, analysis, and conclusions are robust and of the highest standard. This book is a valuable resource for researchers, students, non-governmental organizations, and key decision-makers involved in earthquake disaster management systems at the national, regional, and local levels

Measurements, Models, Systems and Design

531 s.

A Study of Agent Influence in Nested Agent Interactions

This work develops a theory of agent influence and applies it to a coached system of simple reactive agents. Our notion of influence is intended to describe agent ability which is contingent on the actions of other agents and we view such behaviours as being “nested”. An agent may have the ability to make A hold only if another agent has carried out a particular action. Our analysis of this is based on a combination of the observation of the effects of an agent’s actions in a bounded environment and observations on what may be changed in that environment and is intended to allow for a logical representation of nested behaviours. We build on this notion to develop a theory of influence which we offer as an extension of existing systems for representing agency and its effects. The notion of an agent being able to “see to it” that something is brought about has been a useful device for reasoning about agent ability. These so-called STIT semantics have been developed by a number of researchers. Standard STIT semantics allow statements of the form [α stit: A] which says that agent a has the ability to see to it that A holds. Although based on the concept of agent action STIT semantics also allow for the representation of concepts involving what may be thought of as inaction. An agent deciding, for example, not to execute a particular action may be characterised as seeing to it that it does not see to it that A, [α stit: [α stit: -A]]. STIT encourages nesting and although this nesting extends across actions within an agent it does not extend easily across agents. So called other agent statements of the form [β stit: [α stit: A]] do not make sense in standard stit semantics because β seeing to it that α sees to it that A holds implies that β has some dominion over a which, in turn, compromises α’s agency. Although the statement makes no sense under standard STIT it does make sense in an intuitive way and Brian Chellas [31] notes that it would be: “...bizarre to deny that an agent should be able to see to it that another agent sees to something” This is also mentioned in Belnap et al. [8, page 275]. Chellas is correct and there are numerous settings in which other agent STIT does make sense. These settings, which are captured in various readings of STIT, may bring a great deal of system level overhead. In a normative system, for example, β may have the option of imposing a sanction on α if α fails to bring about A and in this sense may be thought of as seeing to it that α sees to it that A holds. Similarly a deontic reading may place β in a position where it is able to place an obligation on α to bring about A. These readings allow for sensible interpretation of other agent STIT but the examples above require that agents have sufficient awareness of personal utility be able to manage sanctions or that they are able to reason about obligations. These readings offer nothing for simple agents with limited resources and abilities. We offer another reading for the STIT element, one based on the concept of agent influence and one which carries minimal system level overhead. Because influence may be contingent on simultaneous or sequential behaviour by a number of agents it is extendible across agents and offers a means of addressing other agent statements. We extend the standard STIT semantics of Horty, Belnap and others with the introduction of “leads to” and “may lead to” operators which allow us to move our analysis into a setting where observation provides evidence of influence. We then explore the manifestation of influence in a number of scenarios. After exploring how influence manifests itself we then offer a partial logical characterisation of the influence operators and discuss its relationship with standard STIT. Building on these semantics and the partial logical characterisation we then explore the practical use of our theory of influence in an agent learning system. We describe experiments with a system specified by safety and liveness properties and having two broad classes of agents, actors and coaches. Actor agents will manipulate their environment and coaching agents will observe the actor’s behaviour and its effects using aggregated observations to generate new behaviours which are then seeded in the environment to modify actor behaviour. We then offer a discussion and evaluation of our theory and its applications indicating where it may be further developed and applied