Evolutionary Neural Gas (ENG): A Model of Self Organizing Network from Input Categorization

Despite their claimed biological plausibility, most self organizing networks have strict topological constraints and consequently they cannot take into account a wide range of external stimuli. Furthermore their evolution is conditioned by deterministic laws which often are not correlated with the structural parameters and the global status of the network, as it should happen in a real biological system. In nature the environmental inputs are noise affected and fuzzy. Which thing sets the problem to investigate the possibility of emergent behaviour in a not strictly constrained net and subjected to different inputs. It is here presented a new model of Evolutionary Neural Gas (ENG) with any topological constraints, trained by probabilistic laws depending on the local distortion errors and the network dimension. The network is considered as a population of nodes that coexist in an ecosystem sharing local and global resources. Those particular features allow the network to quickly adapt to the environment, according to its dimensions. The ENG model analysis shows that the net evolves as a scale-free graph, and justifies in a deeply physical sense- the term gas here used.Comment: 16 pages, 8 figure

Learning shepherding behavior

Roboter, die Schafe hüten sowie die dazu nötigen Strategien zum Bewegen von Individuen zu einem Ziel, bieten vielseitige Anwendungen wie z. B. die Rettung von Menschen aus bedrohlichen Lagen oder der Einsatz schwimmender Roboter zur Beseitigung von Ölteppichen. In dieser Arbeit nutzen wir ein Multiagentensystem als Modell der Roboter und Schafe. Wir untersuchen die Komplexität des Schafehütens und zeigen einen Greedy-Algorithmus, der in linearer Laufzeit eine fast optimale Lösung berechnet. Weiterhin analysieren wir, wie solche Strategien gelernt werden können, da maschinelles Lernen oftmals vorteilhafte Lösungen findet. Im Folgenden nutzen wir Reinforcement Learning (RL) als Lernmethode. Damit RL Agenten ihr gelerntes Wissen auch in kontinuierlichen oder sehr großen Zustandsräumen (wie im betrachteten Szenario) vorhalten können, sind Methoden zur Wissensabstraktion nötig. Unsere Methoden kombinieren RL mit adaptiven neuronalen Verfahren und erlauben dem Agenten gleichzeitig Strategien sowie Darstellungen dieses Wissens zu lernen. Beide Verfahren basieren auf dem unüberwachten Lernverfahren Growing Neural Gas, das eine Vektorquantisierung lernt, indem es neuronale Einheiten im Eingaberaums platziert und bewegt. GNG-Q gruppiert benachbarte Zustände die gleiches Verhalten erfordern (Zustandsraumapproximation); I-GNG-Q wiederum kombiniert Wissen, um eine glatte Bewertungsfunktion zu erhalten (Approximation der Bewertungsfunktion des RL-Agenten). Beide Verfahren beobachten das Verhalten des Lerners um Stellen der Approximation zu finden, die noch verfeinert werden müssen. Die Hauptvorteile unserer Verfahren sind u.a., dass sie ohne Kenntnis des Modells der Umgebung automatisch eine passende Auflösung der Approximation bestimmen. Die experimentelle Analyse unterstreicht, dass unsere Methoden sehr effiziente und effektive Strategien erzeugen.Artificial shepherding strategies, i.e. using robots to move individuals to given locations, have many applications. For example, people can be guided by mobile robots from dangerous places or swimming robots may help to clean up oil spills. This thesis uses a multiagent system to model the robots and sheep. We analyze the complexity of the shepherding task and present a greedy algorithm that only needs linear time to compute a solution that is proven to be close to optimal. Additionally, we analyze to what extend such strategies can be learned as learning usually provides powerful solutions. This thesis focuses on reinforcement learning (RL) as learning method. To enable RL agents to use their knowledge more efficiently in continuous or large state spaces (as e.g. in the shepherding task), methods to transfer knowledge to unseen but similar situations are required. The approaches developed in this thesis, GNG-Q and I-GNG-Q, combine RL with adaptive neural algorithms and enable the agent to learn behavior in parallel with its representation. Both are based upon the growing neural gas, which is an unsupervised learning approach that learns a vector quantization by placing and adjusting units in the input space. GNG-Q groups states that are spatial close and share the same behavior while I-GNG-Q combines the learned behavior from a larger area of the approximation which results in smoother value functions. Thus, GNG-Q performs a state-space abstraction and I-GNG-Q approximates the value function. Both methods monitor the agent's policy during learning to find regions of the approximation that have to be refined. Amongst many others, the core advantages of our approaches are that they do not need the model of the environment and that the resolution of the approximation is determined automatically. The experimental evaluation underlines that the behaviors learned using our approaches are highly efficient and effective.Michael BaumannTag der Verteidigung: 22.01.2016Fakultät für Elektrotechnik, Informatik und Mathematik, Universität Paderborn, Univ., Dissertation, 201

Event-Driven Technologies for Reactive Motion Planning: Neuromorphic Stereo Vision and Robot Path Planning and Their Application on Parallel Hardware

Die Robotik wird immer mehr zu einem Schlüsselfaktor des technischen Aufschwungs. Trotz beeindruckender Fortschritte in den letzten Jahrzehnten, übertreffen Gehirne von Säugetieren in den Bereichen Sehen und Bewegungsplanung noch immer selbst die leistungsfähigsten Maschinen. Industrieroboter sind sehr schnell und präzise, aber ihre Planungsalgorithmen sind in hochdynamischen Umgebungen, wie sie für die Mensch-Roboter-Kollaboration (MRK) erforderlich sind, nicht leistungsfähig genug. Ohne schnelle und adaptive Bewegungsplanung kann sichere MRK nicht garantiert werden. Neuromorphe Technologien, einschließlich visueller Sensoren und Hardware-Chips, arbeiten asynchron und verarbeiten so raum-zeitliche Informationen sehr effizient. Insbesondere ereignisbasierte visuelle Sensoren sind konventionellen, synchronen Kameras bei vielen Anwendungen bereits überlegen. Daher haben ereignisbasierte Methoden ein großes Potenzial, schnellere und energieeffizientere Algorithmen zur Bewegungssteuerung in der MRK zu ermöglichen. In dieser Arbeit wird ein Ansatz zur flexiblen reaktiven Bewegungssteuerung eines Roboterarms vorgestellt. Dabei wird die Exterozeption durch ereignisbasiertes Stereosehen erreicht und die Pfadplanung ist in einer neuronalen Repräsentation des Konfigurationsraums implementiert. Die Multiview-3D-Rekonstruktion wird durch eine qualitative Analyse in Simulation evaluiert und auf ein Stereo-System ereignisbasierter Kameras übertragen. Zur Evaluierung der reaktiven kollisionsfreien Online-Planung wird ein Demonstrator mit einem industriellen Roboter genutzt. Dieser wird auch für eine vergleichende Studie zu sample-basierten Planern verwendet. Ergänzt wird dies durch einen Benchmark von parallelen Hardwarelösungen wozu als Testszenario Bahnplanung in der Robotik gewählt wurde. Die Ergebnisse zeigen, dass die vorgeschlagenen neuronalen Lösungen einen effektiven Weg zur Realisierung einer Robotersteuerung für dynamische Szenarien darstellen. Diese Arbeit schafft eine Grundlage für neuronale Lösungen bei adaptiven Fertigungsprozesse, auch in Zusammenarbeit mit dem Menschen, ohne Einbußen bei Geschwindigkeit und Sicherheit. Damit ebnet sie den Weg für die Integration von dem Gehirn nachempfundener Hardware und Algorithmen in die Industrierobotik und MRK

Automated Reinforcement Learning:An Overview

Reinforcement Learning and recently Deep Reinforcement Learning are popular methods for solving sequential decision making problems modeled as Markov Decision Processes. RL modeling of a problem and selecting algorithms and hyper-parameters require careful considerations as different configurations may entail completely different performances. These considerations are mainly the task of RL experts; however, RL is progressively becoming popular in other fields where the researchers and system designers are not RL experts. Besides, many modeling decisions, such as defining state and action space, size of batches and frequency of batch updating, and number of timesteps are typically made manually. For these reasons, automating different components of RL framework is of great importance and it has attracted much attention in recent years. Automated RL provides a framework in which different components of RL including MDP modeling, algorithm selection and hyper-parameter optimization are modeled and defined automatically. In this article, we explore the literature and present recent work that can be used in automated RL. Moreover, we discuss the challenges, open questions and research directions in AutoRL

Dynamic learning rates for continual unsupervised learning.

The dilemma between stability and plasticity is crucial in machine learning, especially when non-stationary input distributions are considered. This issue can be addressed by continual learning in order to alleviate catastrophic forgetting. This strategy has been previously proposed for supervised and reinforcement learning models. However, little attention has been devoted to unsupervised learning. This work presents a dynamic learning rate framework for unsupervised neural networks that can handle non-stationary distributions. In order for the model to adapt to the input as it changes its characteristics, a varying learning rate that does not merely depend on the training step but on the reconstruction error has been proposed. In the experiments, different configurations for classical competitive neural networks, self-organizing maps and growing neural gas with either per-neuron or per-network dynamic learning rate have been tested. Experimental results on document clustering tasks demonstrate the suitability of the proposal for real-world problems

Gesture-Based Robot Path Shaping

For many individuals, aging is frequently associated with diminished mobility and dexterity. Such decreases may be accompanied by a loss of independence, increased burden to caregivers, or institutionalization. It is foreseen that the ability to retain independence and quality of life as one ages will increasingly depend on environmental sensing and robotics which facilitate aging in place. The development of ubiquitous sensing strategies in the home underpins the promise of adaptive services, assistive robotics, and architectural design which would support a person\u27s ability to live independently as they age. Instrumentation (sensors and processing) which is capable of recognizing the actions and behavioral patterns of an individual is key to the effective component design in these areas. Recognition of user activity and the inference of user intention may be used to inform the action plans of support systems and service robotics within the environment. Automated activity recognition involves detection of events in a sensor data stream, conversion to a compact format, and classification as one of a known set of actions. Once classified, an action may be used to elicit a specific response from those systems designed to provide support to the user. It is this response that is the ultimate use of recognized activity. Hence, the activity may be considered as a command to the system. Extending this concept, a set of distinct activities in the form of hand and arm gestures may form the basis of a command interface for human-robot interaction. A gesture-based interface of this type promises an intuitive method for accessing computing and other assistive resources so as to promote rapid adoption by elderly, impaired, or otherwise unskilled users. This thesis includes a thorough survey of relevant work in the area of machine learning for activity and gesture recognition. Previous approaches are compared for their relative benefits and limitations. A novel approach is presented which utilizes user-generated feedback to rate the desirability of a robotic response to gesture. Poorly rated responses are altered so as to elicit improved ratings on subsequent observations. In this way, responses are honed toward increasing effectiveness. A clustering method based on the Growing Neural Gas (GNG) algorithm is used to create a topological map of reference nodes representing input gesture types. It is shown that learning of desired responses to gesture may be accelerated by exploiting well-rewarded actions associated with reference nodes in a local neighborhood of the growing neural gas topology. Significant variation in the user\u27s performance of gestures is interpreted as a new gesture for which the system must learn a desired response. A method for allowing the system to learn new gestures while retaining past training is also proposed and shown to be effective

A survey on machine learning for recurring concept drifting data streams

The problem of concept drift has gained a lot of attention in recent years. This aspect is key in many domains exhibiting non-stationary as well as cyclic patterns and structural breaks affecting their generative processes. In this survey, we review the relevant literature to deal with regime changes in the behaviour of continuous data streams. The study starts with a general introduction to the field of data stream learning, describing recent works on passive or active mechanisms to adapt or detect concept drifts, frequent challenges in this area, and related performance metrics. Then, different supervised and non-supervised approaches such as online ensembles, meta-learning and model-based clustering that can be used to deal with seasonalities in a data stream are covered. The aim is to point out new research trends and give future research directions on the usage of machine learning techniques for data streams which can help in the event of shifts and recurrences in continuous learning scenarios in near real-time

An incremental clustering and associative learning architecture for intelligent robotics

The ability to learn from the environment and memorise the acquired knowledge is essential for robots to become autonomous and versatile artificial companions. This thesis proposes a novel learning and memory architecture for robots, which performs associative learning and recall of sensory and actuator patterns. The approach avoids the inclusion of task-specific expert knowledge and can deal with any kind of multi-dimensional real-valued data, apart from being tolerant to noise and supporting incremental learning. The proposed architecture integrates two machine learning methods: a topology learning algorithm that performs incremental clustering, and an associative memory model that learns relationship information based on the co-occurrence of inputs. The evaluations of both the topology learning algorithm and the associative memory model involved the memorisation of high-dimensional visual data as well as the association of symbolic data, presented simultaneously and sequentially. Moreover, the document analyses the results of two experiments in which the entire architecture was evaluated regarding its associative and incremental learning capabilities. One experiment comprised an incremental learning task with visual patterns and text labels, which was performed both in a simulated scenario and with a real robot. In a second experiment a robot learned to recognise visual patterns in the form of road signs and associated them with di erent con gurations of its arm joints. The thesis also discusses several learning-related aspects of the architecture and highlights strengths and weaknesses of the proposed approach. The developed architecture and corresponding ndings contribute to the domains of machine learning and intelligent robotics