Design and process/measurement for immersed element control in a reconfigurable vertically falling soap film

Thesis (S.B.)--Massachusetts Institute of Technology, Dept. of Mechanical Engineering, 2007.Includes bibliographical references (p. 24-25).Reinforcement learning has proven successful at harnessing the passive dynamics of underactuated systems to achieve least energy solutions. However, coupled fluid-structural models are too computationally intensive for in-the-loop control in viscous flow regimes. My vertically falling soap film will provide a reconfigurable experimental environment for machine learning controllers. The real-time position and velocity data will be collected with a High Speed Video system, illuminated by a Low Pressure Sodium Lamp. Approximating lines of interference within the soap film to known pressure variations, controllers will shape downstream flow to desired conditions. Though accurate measurement still eludes those without Laser Doppler Velocimetry, order of magnitude Reynolds numbers can be estimated to describe the regime of controller inquiry.by John Glowa.S.B

Biologically inspired learning system

Learning Systems used on robots require either a-priori knowledge in the form of models, rules of thumb or databases or require that robot to physically execute multitudes of trial solutions. The first requirement limits the robot’s ability to operate in unstructured changing environments, and the second limits the robot’s service life and resources. In this research a generalized approach to learning was developed through a series of algorithms that can be used for construction of behaviors that are able to cope with unstructured environments through adaptation of both internal parameters and system structure as a result of a goal based supervisory mechanism. Four main learning algorithms have been developed, along with a goal directed random exploration routine. These algorithms all use the concept of learning from a recent memory in order to save the robot/agent from having to exhaustively execute all trial solutions. The first algorithm is a reactive online learning algorithm that uses a supervised learning to find the sensor/action combinations that promote realization of a preprogrammed goal. It produces a feed forward neural network controller that is used to control the robot. The second algorithm is similar to first in that it uses a supervised learning strategy, but it produces a neural network that considers past values, thus providing a non-reactive solution. The third algorithm is a departure from the first two in that uses a non-supervised learning technique to learn the best actions for each situation the robot encounters. The last algorithm builds a graph of the situations encountered by agent/robot in order to learn to associate the best actions with sensor inputs. It uses an unsupervised learning approach based on shortest paths to a goal situation in the graph in order to generate a non-reactive feed forward neural network. Test results were good, the first and third algorithms were tested in a formation maneuvering task in both simulation and onboard mobile robots, while the second and fourth were tested simulation

Application of Fuzzy State Aggregation and Policy Hill Climbing to Multi-Agent Systems in Stochastic Environments

Reinforcement learning is one of the more attractive machine learning technologies, due to its unsupervised learning structure and ability to continually even as the operating environment changes. Applying this learning to multiple cooperative software agents (a multi-agent system) not only allows each individual agent to learn from its own experience, but also opens up the opportunity for the individual agents to learn from the other agents in the system, thus accelerating the rate of learning. This research presents the novel use of fuzzy state aggregation, as the means of function approximation, combined with the policy hill climbing methods of Win or Lose Fast (WoLF) and policy-dynamics based WoLF (PD-WoLF). The combination of fast policy hill climbing (PHC) and fuzzy state aggregation (FSA) function approximation is tested in two stochastic environments; Tileworld and the robot soccer domain, RoboCup. The Tileworld results demonstrate that a single agent using the combination of FSA and PHC learns quicker and performs better than combined fuzzy state aggregation and Q-learning lone. Results from the RoboCup domain again illustrate that the policy hill climbing algorithms perform better than Q-learning alone in a multi-agent environment. The learning is further enhanced by allowing the agents to share their experience through a weighted strategy sharing

Development of Cognitive Capabilities in Humanoid Robots

Merged with duplicate record 10026.1/645 on 03.04.2017 by CS (TIS)Building intelligent systems with human level of competence is the ultimate grand challenge for science and technology in general, and especially for the computational intelligence community. Recent theories in autonomous cognitive systems have focused on the close integration (grounding) of communication with perception, categorisation and action. Cognitive systems are essential for integrated multi-platform systems that are capable of sensing and communicating. This thesis presents a cognitive system for a humanoid robot that integrates abilities such as object detection and recognition, which are merged with natural language understanding and refined motor controls. The work includes three studies; (1) the use of generic manipulation of objects using the NMFT algorithm, by successfully testing the extension of the NMFT to control robot behaviour; (2) a study of the development of a robotic simulator; (3) robotic simulation experiments showing that a humanoid robot is able to acquire complex behavioural, cognitive, and linguistic skills through individual and social learning. The robot is able to learn to handle and manipulate objects autonomously, to cooperate with human users, and to adapt its abilities to changes in internal and environmental conditions. The model and the experimental results reported in this thesis, emphasise the importance of embodied cognition, i.e. the humanoid robot's physical interaction between its body and the environment

Viability in State-Action Space: Connecting Morphology, Control, and Learning

Wie können wir Robotern ermöglichen, modellfrei und direkt auf der Hardware zu lernen? Das maschinelle Lernen nimmt als Standardwerkzeug im Arsenal des Robotikers seinen Platz ein. Es gibt jedoch einige offene Fragen, wie man die Kontrolle über physikalische Systeme lernen kann. Diese Arbeit gibt zwei Antworten auf diese motivierende Frage. Das erste ist ein formales Mittel, um die inhärente Robustheit eines gegebenen Systemdesigns zu quantifizieren, bevor der Controller oder das Lernverfahren entworfen wird. Dies unterstreicht die Notwendigkeit, sowohl das Hardals auch das Software-Design eines Roboters zu berücksichtigen, da beide Aspekte in der Systemdynamik untrennbar miteinander verbunden sind. Die zweite ist die Formalisierung einer Sicherheitsmass, die modellfrei erlernt werden kann. Intuitiv zeigt diese Mass an, wie leicht ein Roboter Fehlschläge vermeiden kann. Auf diese Weise können Roboter unbekannte Umgebungen erkunden und gleichzeitig Ausfälle vermeiden. Die wichtigsten Beiträge dieser Dissertation basieren sich auf der Viabilitätstheorie. Viabilität bietet eine alternative Sichtweise auf dynamische Systeme: Anstatt sich auf die Konvergenzeigenschaften eines Systems in Richtung Gleichgewichte zu konzentrieren, wird der Fokus auf Menge von Fehlerzuständen und die Fähigkeit des Systems, diese zu vermeiden, verlagert. Diese Sichtweise eignet sich besonders gut für das Studium der Lernkontrolle an Robotern, da Stabilität im Sinne einer Konvergenz während des Lernprozesses selten gewährleistet werden kann. Der Begriff der Viabilität wird formal auf den Zustand-Aktion-Raum erweitert, mit Viabilitätsmengen von Staat-Aktionspaaren. Eine über diese Mengen definierte Mass ermöglicht eine quantifizierte Bewertung der Robustheit, die für die Familie aller fehlervermeidenden Regler gilt, und ebnet den Weg für ein sicheres, modellfreies Lernen. Die Arbeit beinhaltet auch zwei kleinere Beiträge. Der erste kleine Beitrag ist eine empirische Demonstration der Shaping durch ausschliessliche Modifikation der Systemdynamik. Diese Demonstration verdeutlicht die Bedeutung der Robustheit gegenüber Fehlern für die Lernkontrolle: Ausfälle können nicht nur Schäden verursachen, sondern liefern in der Regel auch keine nützlichen Gradienteninformationen für den Lernprozess. Der zweite kleine Beitrag ist eine Studie über die Wahl der Zustandsinitialisierungen. Entgegen der Intuition und der üblichen Praxis zeigt diese Studie, dass es zuverlässiger sein kann, das System gelegentlich aus einem Zustand zu initialisieren, der bekanntermassen unkontrollierbar ist.How can we enable robots to learn control model-free and directly on hardware? Machine learning is taking its place as a standard tool in the roboticist’s arsenal. However, there are several open questions on how to learn control for physical systems. This thesis provides two answers to this motivating question. The first is a formal means to quantify the inherent robustness of a given system design, prior to designing the controller or learning agent. This emphasizes the need to consider both the hardware and software design of a robot, which are inseparably intertwined in the system dynamics. The second is the formalization of a safety-measure, which can be learned model-free. Intuitively, this measure indicates how easily a robot can avoid failure, and enables robots to explore unknown environments while avoiding failures. The main contributions of this dissertation are based on viability theory. Viability theory provides a slightly unconventional view of dynamical systems: instead of focusing on a system’s convergence properties towards equilibria, the focus is shifted towards sets of failure states and the system’s ability to avoid these sets. This view is particularly well suited to studying learning control in robots, since stability in the sense of convergence can rarely be guaranteed during the learning process. The notion of viability is formally extended to state-action space, with viable sets of state-action pairs. A measure defined over these sets allows a quantified evaluation of robustness valid for the family of all failure-avoiding control policies, and also paves the way for enabling safe model-free learning. The thesis also includes two minor contributions. The first minor contribution is an empirical demonstration of shaping by exclusively modifying the system dynamics. This demonstration highlights the importance of robustness to failures for learning control: not only can failures cause damage, but they typically do not provide useful gradient information for the learning process. The second minor contribution is a study on the choice of state initializations. Counter to intuition and common practice, this study shows it can be more reliable to occasionally initialize the system from a state that is known to be uncontrollable

Online evolution of robot behaviour

Tese de mestrado em Engenharia Informática (Interação e Conhecimento), apresentada à Universidade de Lisboa, através da Faculdade de Ciências, 2012In this dissertation, we propose and evaluate two novel approaches to the online synthesis of neural controllers for autonomous robots. The first approach is odNEAT, an online, distributed, and decentralized version of NeuroEvolution of Augmenting Topologies (NEAT). odNEAT is an algorithm for online evolution in groups of embodied agents such as robots. In odNEAT, agents have to solve the same task, either individually or collectively. While previous approaches to online evolution of neural controllers have been limited to the optimization of weights, odNEAT evolves both weights and network topology. We demonstrate odNEAT through a series of simulation-based experiments in which a group of e-puck-like robots must perform an aggregation task. Our results show that robots are capable of evolving effective aggregation strategies and that sustainable behaviours evolve quickly. We show that odNEAT approximates the performance of rtNEAT, a similar but centralized method. We also analyze the contribution of each algorithmic component on the performance through a series of ablation studies. In the second approach, we extend our previous method and combine online evolution of weights and network topology (odNEAT) with neuromodulated learning. We demonstrate our method through a series of experiments in which a group of simulated robots must perform a dynamic concurrent foraging task. In this task, scattered food items periodically change their nutritive value or become poisonous. Our results show that when neuromodulated learning is employed, neural controllers are synthesized faster than by odNEAT alone. We demonstrate that the online evolutionary process is capable of generating controllers that adapt to the periodic task changes. We evaluate the performance both in a single robot setup and in a multirobot setup. An analysis of the evolved networks shows that they are characterized by specialized modulatory neurons that exclusively regulate online learning in the output neurons

Applied Analysis and Synthesis of Complex Systems: Proceedings of the IIASA-Kyoto University Joint Seminar, June 28-29, 2004

This two-day seminar aimed at introducing the new development of the COE by Kyoto University to IIASA and discussing general modeling methodologies for complex systems consisting of many elements, mostly via nonlinear, large-scale interactions. We aimed at clarifying fundamental principles in complex phenomena as well as utilizing and synthesizing the knowledge derived out of them. The 21st Century COE (Center of Excellence) Program is an initiative by the Japanese Ministry of Education, Culture, Science and Technology (MEXT) to support universities establishing discipline-specific international centers for education and research, and to enhance the universities to be the world's apex of excellence with international competitiveness in the specific research areas. Our program of "Research and Education on Complex Functional Mechanical Systems" is successfully selected to be awarded the fund for carrying out new research and education as Centers of Excellence in the field of mechanical engineering in 2003 (five-year project), and is expected to lead Japanese research and education, and endeavor to be the top in the world. The program covers general backgrounds in diverse fields as well as a more in-depth grasp of specific branches such as complex system modeling and analysis of the problems including: nonlinear dynamics, micro-mesoscopic physics, turbulent transport phenomena, atmosphere-ocean systems, robots, human-system interactions, and behaviors of nano-composites and biomaterials. Fundamentals of those complex functional mechanical systems are macroscopic phenomena of complex systems consisting of microscopic elements, mostly via nonlinear, large-scale interactions, which typically present collective behavior such as self-organization, pattern formation, etc. Such phenomena can be observed or created in every aspect of modern technologies. Especially, we are focusing upon; turbulent transport phenomena in climate modeling, dynamical and chaotic behaviors in control systems and human-machine systems, and behaviors of mechanical materials with complex structures. As a partial attainment of this program, IIASA and Kyoto University have exchanged Consortia Agreement at the beginning of the program in 2003, and this seminar was held to introduce the outline of the COE program of Kyoto University to IIASA researchers and to deepen the shared understandings on novel complex system modeling and analysis, including novel climate modeling and carbonic cycle management, through joint academic activities by mechanical engineers and system engineers. In this seminar, we invited a distinguished researcher in Europe as a keynote speaker and our works attained so far in the project were be presented by the core members of the project as well as by the other contributing members who participated in the project. All IIASA research staff and participants of YSSP (Young Scientist Summer Program) were cordially invited to attend this seminar to discuss general modeling methodologies for complex systems