Lifelong Reinforcement Learning On Mobile Robots

Machine learning has shown tremendous growth in the past decades, unlocking new capabilities in a variety of fields including computer vision, natural language processing, and robotic control. While the sophistication of individual problems a learning system can handle has greatly advanced, the ability of a system to extend beyond an individual problem to adapt and solve new problems has progressed more slowly. This thesis explores the problem of progressive learning. The goal is to develop methodologies that accumulate, transfer, and adapt knowledge in applied settings where the system is faced with the ambiguity and resource limitations of operating in the physical world. There are undoubtedly many challenges to designing such a system, my thesis looks at the component of this problem related to how knowledge from previous tasks can be a benefit in the domain of reinforcement learning where the agent receives rewards for positive actions. Reinforcement learning is particularly difficult when training on physical systems, like mobile robots, where repeated trials can damage the system and unrestricted exploration is often associated with safety risks. I investigate how knowledge can be efficiently accumulated and applied to future reinforcement learning problems on mobile robots in order to reduce sample complexity and enable systems to adapt to novel settings. Doing this involves mathematical models which can combine knowledge from multiple tasks, methods for restructuring optimizations and data collection to handle sequential updates, and data selection strategies that can be used to address resource limitations

Development of Self-Learning Type-2 Fuzzy Systems for System Identification and Control of Autonomous Systems

Modelling and control of dynamic systems are faced by multiple technical challenges, mainly due to the nature of uncertain complex, nonlinear, and time-varying systems. Traditional modelling techniques require a complete understanding of system dynamics and obtaining comprehensive mathematical models is not always achievable due to limited knowledge of the systems as well as the presence of multiple uncertainties in the environment. As universal approximators, fuzzy logic systems (FLSs), neural networks (NNs) and neuro-fuzzy systems have proved to be successful computational tools for representing the behaviour of complex dynamical systems. Moreover, FLSs, NNs and learning-based techniques have been gaining popularity for controlling complex, ill-defined, nonlinear, and time-varying systems in the face of uncertainties. However, fuzzy rules derived by experts can be too ad-hoc, and the performance is less than optimum. In other words, generating fuzzy rules and membership functions in fuzzy systems is a potential challenge especially for systems with many variables. Moreover, under the umbrella of FLSs, although type-1 fuzzy logic control systems (T1-FLCs) have been applied to control various complex nonlinear systems, they have limited capability to handle uncertainties. Aiming to accommodate uncertainties, type-2 fuzzy logic control systems (T2-FLCs) were established. This thesis aims to address the shortcomings of existing fuzzy techniques by utilisation of type-2 FLCs with novel adaptive capabilities. The first contribution of this thesis is a novel online system identification technique by means of a recursive interval type-2 Takagi-Sugeno fuzzy C-means clustering technique (IT2-TS-FC) to accommodate the footprint-of-uncertainties (FoUs). This development is meant to specifically address the shortcomings of type-1 fuzzy systems in capturing the footprint-of-uncertainties such as mechanical wear, rotor damage, battery drain and sensor and actuator faults. Unlike previous type-2 TS fuzzy models, the proposed method constructs two fuzzifiers (upper and lower) and two regression coefficients in the consequent part to handle uncertainties. The weighted least square method is employed to compute the regression coefficients. The proposed method is validated using two benchmarks, namely, real flight test data of a quadcopter drone and Mackey-Glass time series data. The algorithm has the capability to model uncertainties (e.g., noisy dataset). The second contribution of this thesis is the development of a novel self-adaptive interval type-2 fuzzy controller named the SAF2C for controlling multi-input multi-output (MIMO) nonlinear systems. The adaptation law is derived using sliding mode control (SMC) theory to reduce the computation time so that the learning process can be expedited by 80% compared to separate single-input single-output (SISO) controllers. The system employs the `Enhanced Iterative Algorithm with Stop Condition' (EIASC) type-reduction method, which is more computationally efficient than the `Karnik-Mendel' type-reduction algorithm. The stability of the SAF2C is proven using the Lyapunov technique. To ensure the applicability of the proposed control scheme, SAF2C is implemented to control several dynamical systems, including a simulated MIMO hexacopter unmanned aerial vehicle (UAV) in the face of external disturbance and parameter variations. The ability of SAF2C to filter the measurement noise is demonstrated, where significant improvement is obtained using the proposed controller in the face of measurement noise. Also, the proposed closed-loop control system is applied to control other benchmark dynamic systems (e.g., a simulated autonomous underwater vehicle and inverted pendulum on a cart system) demonstrating high accuracy and robustness to variations in system parameters and external disturbance. Another contribution of this thesis is a novel stand-alone enhanced self-adaptive interval type-2 fuzzy controller named the ESAF2C algorithm, whose type-2 fuzzy parameters are tuned online using the SMC theory. This way, we expect to design a computationally efficient adaptive Type-2 fuzzy system, suitable for real-time applications by introducing the EIASC type-reducer. The proposed technique is applied on a quadcopter UAV (QUAV), where extensive simulations and real-time flight tests for a hovering QUAV under wind disturbances are also conducted to validate the efficacy of the ESAF2C. Specifically, the control performance is investigated in the face of external wind gust disturbances, generated using an industrial fan. Stability analysis of the ESAF2C control system is investigated using the Lyapunov theory. Yet another contribution of this thesis is the development of a type-2 evolving fuzzy control system (T2-EFCS) to facilitate self-learning (either from scratch or from a certain predefined rule). T2-EFCS has two phases, namely, the structure learning and the parameters learning. The structure of T2-EFCS does not require previous information about the fuzzy structure, and it can start the construction of its rules from scratch with only one rule. The rules are then added and pruned in an online fashion to achieve the desired set-point. The proposed technique is applied to control an unmanned ground vehicle (UGV) in the presence of multiple external disturbances demonstrating the robustness of the proposed control systems. The proposed approach turns out to be computationally efficient as the system employs fewer fuzzy parameters while maintaining superior control performance

Employment 5.0: the work of the future and the future of work

Additional research group involved: Centre for Computing and Social Responsibility open access articleThis systematic review brings together the collection of recent scholarly outputs on the disruptive impact of digital transformation on the work. This paper draws from a sample of 68 outputs from 2011 to 2022. We identify three key theoretical perspectives: socio-technical systems theory, skill-biased technological change, and political economy of digital transformation. The articles provide complementary insights on cross-cutting themes of technological unemployment, wage inequality and job polarisation. They also highlight often conflicting views about technology ownership, work-less utopia, education reforms and the imperative of human-centricity in appropriation of technology. Drawing on the findings across the whole spectrum of theoretical and analytical perspectives, we offer critical reflections about the factors that will define the work of the future, in terms of skills, creativity and opportunities for autonomous workers. We also discuss the political and institutional processes that will shape the future of work. Finally, we offer recommendations for future research and policy interventions

Sensing and Control for Robust Grasping with Simple Hardware

Robots can move, see, and navigate in the real world outside carefully structured factories, but they cannot yet grasp and manipulate objects without human intervention. Two key barriers are the complexity of current approaches, which require complicated hardware or precise perception to function effectively, and the challenge of understanding system performance in a tractable manner given the wide range of factors that impact successful grasping. This thesis presents sensors and simple control algorithms that relax the requirements on robot hardware, and a framework to understand the capabilities and limitations of grasping systems.Engineering and Applied Science

DEVELOPMENT OF A ROBOTIC EXOSKELETON SYSTEM FOR GAIT REHABILITATION

Ph.DDOCTOR OF PHILOSOPH

Sustainable Technology and Elderly Life

The coming years will see an exponential increase in the proportion of elderly people in our society. This accelerated growth brings with it major challenges in relation to the sustainability of the system. There are different aspects where these changes will have a special incidence: health systems and their monitoring; the development of a framework in which the elderly can develop their daily lives satisfactorily; and in the design of intelligent cities adapted to the future sociodemographic profile. The discussion of the challenges faced, together with the current technological evolution, can show possible ways of meeting the challenges. There are different aspects where these changes will have a special incidence: health systems and their monitoring; the development of a framework in which the elderly can develop their daily lives satisfactorily; and in the design of intelligent cities adapted to the future sociodemographic profile. This special issue discusses various ways in which sustainable technologies can be applied to improve the lives of the elderly. Six articles on the subject are featured in this volume. From a systematic review of the literature to the development of gamification and health improvement projects. The articles present suggestive proposals for the improvement of the lives of the elderly. The volume is a resource of interest for the scientific community, since it shows different research gaps in the current state of the art. But it is also a document that can help social policy makers and people working in this domain to planning successful projects

Program and Abstracts Celebration of Student Scholarship, 2016

Program and Abstracts from the Celebration of Student Scholarship on April 27, 2016

Makers at School, Educational Robotics and Innovative Learning Environments

This open access book contains observations, outlines, and analyses of educational robotics methodologies and activities, and developments in the field of educational robotics emerging from the findings presented at FabLearn Italy 2019, the international conference that brought together researchers, teachers, educators and practitioners to discuss the principles of Making and educational robotics in formal, non-formal and informal education. The editors’ analysis of these extended versions of papers presented at FabLearn Italy 2019 highlight the latest findings on learning models based on Making and educational robotics. The authors investigate how innovative educational tools and methodologies can support a novel, more effective and more inclusive learner-centered approach to education. The following key topics are the focus of discussion: Makerspaces and Fab Labs in schools, a maker approach to teaching and learning; laboratory teaching and the maker approach, models, methods and instruments; curricular and non-curricular robotics in formal, non-formal and informal education; social and assistive robotics in education; the effect of innovative spaces and learning environments on the innovation of teaching, good practices and pilot projects

Fast Sensing and Adaptive Actuation for Robust Legged Locomotion

Robust legged locomotion in complex terrain demands fast perturbation detection and reaction. In animals, due to the neural transmission delays, the high-level control loop involving the brain is absent from mitigating the initial disturbance. Instead, the low-level compliant behavior embedded in mechanics and the mid-level controllers in the spinal cord are believed to provide quick response during fast locomotion. Still, it remains unclear how these low- and mid-level components facilitate robust locomotion. This thesis aims to identify and characterize the underlining elements responsible for fast sensing and actuation. To test individual elements and their interplay, several robotic systems were implemented. The implementations include active and passive mechanisms as a combination of elasticities and dampers in multi-segment robot legs, central pattern generators inspired by intraspinal controllers, and a synthetic robotic version of an intraspinal sensor. The first contribution establishes the notion of effective damping. Effective damping is defined as the total energy dissipation during one step, which allows quantifying how much ground perturbation is mitigated. Using this framework, the optimal damper is identified as viscous and tunable. This study paves the way for integrating effective dampers to legged designs for robust locomotion. The second contribution introduces a novel series elastic actuation system. The proposed system tackles the issue of power transmission over multiple joints, while featuring intrinsic series elasticity. The design is tested on a hopper with two more elastic elements, demonstrating energy recuperation and enhanced dynamic performance. The third contribution proposes a novel tunable damper and reveals its influence on legged hopping. A bio-inspired slack tendon mechanism is implemented in parallel with a spring. The tunable damping is rigorously quantified on a central-pattern-generator-driven hopping robot, which reveals the trade-off between locomotion robustness and efficiency. The last contribution explores the intraspinal sensing hypothesis of birds. We speculate that the observed intraspinal structure functions as an accelerometer. This accelerometer could provide fast state feedback directly to the adjacent central pattern generator circuits, contributing to birds’ running robustness. A biophysical simulation framework is established, which provides new perspectives on the sensing mechanics of the system, including the influence of morphologies and material properties. Giving an overview of the hierarchical control architecture, this thesis investigates the fast sensing and actuation mechanisms in several control layers, including the low-level mechanical response and the mid-level intraspinal controllers. The contributions of this work provide new insight into animal loco-motion robustness and lays the foundation for future legged robot design