Iterative learning control for impulsive multi-agent systems with varying trial lengths

In this paper, we introduce iterative learning control (ILC) schemes with varying trial lengths (VTL) to control impulsive multi-agent systems (I-MAS). We use domain alignment operator to characterize each tracking error to ensure that the error can completely update the control function during each iteration. Then we analyze the system’s uniform convergence to the target leader. Further, we use two local average operators to optimize the control function such that it can make full use of the iteration error. Finally, numerical examples are provided to verify the theoretical results

Iterative learning of human partner's desired trajectory for proactive human-robot collaboration

A period-varying iterative learning control scheme is proposed for a robotic manipulator to learn a target trajectory that is planned by a human partner but unknown to the robot, which is a typical scenario in many applications. The proposed method updates the robot’s reference trajectory in an iterative manner to minimize the interaction force applied by the human. Although a repetitive human–robot collaboration task is considered, the task period is subject to uncertainty introduced by the human. To address this issue, a novel learning mechanism is proposed to achieve the control objective. Theoretical analysis is performed to prove the performance of the learning algorithm and robot controller. Selective simulations and experiments on a robotic arm are carried out to show the effectiveness of the proposed method in human–robot collaboration

Model-Based Policy Search for Automatic Tuning of Multivariate PID Controllers

PID control architectures are widely used in industrial applications. Despite their low number of open parameters, tuning multiple, coupled PID controllers can become tedious in practice. In this paper, we extend PILCO, a model-based policy search framework, to automatically tune multivariate PID controllers purely based on data observed on an otherwise unknown system. The system's state is extended appropriately to frame the PID policy as a static state feedback policy. This renders PID tuning possible as the solution of a finite horizon optimal control problem without further a priori knowledge. The framework is applied to the task of balancing an inverted pendulum on a seven degree-of-freedom robotic arm, thereby demonstrating its capabilities of fast and data-efficient policy learning, even on complex real world problems.Comment: Accepted final version to appear in 2017 IEEE International Conference on Robotics and Automation (ICRA

Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Design and development of the ‘POD Adventures’ smartphone game: a blended problem-solving intervention for adolescent mental health in India

Introduction: Digital technology platforms offer unparalleled opportunities to reach vulnerable adolescents at scale and overcome many barriers that exist around conventional service provision. This paper describes the design and development of POD Adventures, a blended problem-solving game-based intervention for adolescents with or at risk of anxiety, depression and conduct difficulties in India. This intervention was developed as part of the PRemIum for ADolEscents (PRIDE) research programme, which aims to establish a suite of transdiagnostic psychological interventions organised around a stepped care system in Indian secondary schools. Methods and materials: Intervention development followed a person-centered approach consisting of four iterative activities: (i) review of recent context-specific evidence on mental health needs and preferences for the target population of school-going Indian adolescents, including a multiple stakeholder analysis of school counselling priorities and pilot studies of a brief problem-solving intervention; (ii) new focus group discussions with N=46 student participants and N=8 service providers; (iii) co-design workshops with N=22 student participants and N=8 service providers; and (iv) user-testing with N=50 student participants. Participants were aged 12-17 years and recruited from local schools in New Delhi and Goa, including a subgroup with self-identified mental health needs (N=6). Results: Formative data from existing primary sources, new focus groups and co-design workshops supported a blended format for delivering a brief problem-solving intervention, with counsellors supporting use of a game-based app on ‘offline’ smartphones. User-testing with prototypes identified a need for simplification of language, use of concrete examples of concepts and practice elements to enhance engagement. There were also indications that participants most valued relatability and interactivity within real-world stories with judicious support from an in-app guide. The final prototype comprised a set of interactive and gamified vignettes and a structured set of problem-solving questions to consolidate and generalise learning while encouraging real-world application. Discussion: Findings shaped the design of POD Adventures and its delivery as an open-access blended intervention for secondary school students with a felt need for psychological support, consistent with an early intervention paradigm. A randomised controlled trial is planned to evaluate processes and impacts of POD Adventures when delivered for help-seeking students in low-resource school settings

ON ITERATIVE LEARNING CONTROL FOR SOLVING NEW CONTROL PROBLEMS

Ph.DDOCTOR OF PHILOSOPH

Spatio-temporal learning with the online finite and infinite echo-state Gaussian processes

Successful biological systems adapt to change. In this paper, we are principally concerned with adaptive systems that operate in environments where data arrives sequentially and is multivariate in nature, for example, sensory streams in robotic systems. We contribute two reservoir inspired methods: 1) the online echostate Gaussian process (OESGP) and 2) its infinite variant, the online infinite echostate Gaussian process (OIESGP) Both algorithms are iterative fixed-budget methods that learn from noisy time series. In particular, the OESGP combines the echo-state network with Bayesian online learning for Gaussian processes. Extending this to infinite reservoirs yields the OIESGP, which uses a novel recursive kernel with automatic relevance determination that enables spatial and temporal feature weighting. When fused with stochastic natural gradient descent, the kernel hyperparameters are iteratively adapted to better model the target system. Furthermore, insights into the underlying system can be gleamed from inspection of the resulting hyperparameters. Experiments on noisy benchmark problems (one-step prediction and system identification) demonstrate that our methods yield high accuracies relative to state-of-the-art methods, and standard kernels with sliding windows, particularly on problems with irrelevant dimensions. In addition, we describe two case studies in robotic learning-by-demonstration involving the Nao humanoid robot and the Assistive Robot Transport for Youngsters (ARTY) smart wheelchair