From Knowing to Doing: Learning Diverse Motor Skills through Instruction Learning

Recent years have witnessed many successful trials in the robot learning field. For contact-rich robotic tasks, it is challenging to learn coordinated motor skills by reinforcement learning. Imitation learning solves this problem by using a mimic reward to encourage the robot to track a given reference trajectory. However, imitation learning is not so efficient and may constrain the learned motion. In this paper, we propose instruction learning, which is inspired by the human learning process and is highly efficient, flexible, and versatile for robot motion learning. Instead of using a reference signal in the reward, instruction learning applies a reference signal directly as a feedforward action, and it is combined with a feedback action learned by reinforcement learning to control the robot. Besides, we propose the action bounding technique and remove the mimic reward, which is shown to be crucial for efficient and flexible learning. We compare the performance of instruction learning with imitation learning, indicating that instruction learning can greatly speed up the training process and guarantee learning the desired motion correctly. The effectiveness of instruction learning is validated through a bunch of motion learning examples for a biped robot and a quadruped robot, where skills can be learned typically within several million steps. Besides, we also conduct sim-to-real transfer and online learning experiments on a real quadruped robot. Instruction learning has shown great merits and potential, making it a promising alternative for imitation learning

Data science, analytics and artificial intelligence in e-health : trends, applications and challenges

Acknowledgments. This work has been partially supported by the Divina Pastora Seguros company.More than ever, healthcare systems can use data, predictive models, and intelligent algorithms to optimize their operations and the service they provide. This paper reviews the existing literature regarding the use of data science/analytics methods and artificial intelligence algorithms in healthcare. The paper also discusses how healthcare organizations can benefit from these tools to efficiently deal with a myriad of new possibilities and strategies. Examples of real applications are discussed to illustrate the potential of these methods. Finally, the paper highlights the main challenges regarding the use of these methods in healthcare, as well as some open research lines

Model Predictive Control (MPC) for Enhancing Building and HVAC System Energy Efficiency: Problem Formulation, Applications and Opportunities

In the last few years, the application of Model Predictive Control (MPC) for energy management in buildings has received significant attention from the research community. MPC is becoming more and more viable because of the increase in computational power of building automation systems and the availability of a significant amount of monitored building data. MPC has found successful implementation in building thermal regulation, fully exploiting the potential of building thermal mass. Moreover, MPC has been positively applied to active energy storage systems, as well as to the optimal management of on-site renewable energy sources. MPC also opens up several opportunities for enhancing energy efficiency in the operation of Heating Ventilation and Air Conditioning (HVAC) systems because of its ability to consider constraints, prediction of disturbances and multiple conflicting objectives, such as indoor thermal comfort and building energy demand. Despite the application of MPC algorithms in building control has been thoroughly investigated in various works, a unified framework that fully describes and formulates the implementation is still lacking. Firstly, this work introduces a common dictionary and taxonomy that gives a common ground to all the engineering disciplines involved in building design and control. Secondly the main scope of this paper is to define the MPC formulation framework and critically discuss the outcomes of different existing MPC algorithms for building and HVAC system management. The potential benefits of the application of MPC in improving energy efficiency in buildings were highlighted

Transport systems analysis : models and data

Funding: This research project has been funded by Spanish R+D Programs, specifcally under Grant PID2020-112967GB-C31.Rapid advancements in new technologies, especially information and communication technologies (ICT), have significantly increased the number of sensors that capture data, namely those embedded in mobile devices. This wealth of data has garnered particular interest in analyzing transport systems, with some researchers arguing that the data alone are sufficient enough to render transport models unnecessary. However, this paper takes a contrary position and holds that models and data are not mutually exclusive but rather depend upon each other. Transport models are built upon established families of optimization and simulation approaches, and their development aligns with the scientific principles of operations research, which involves acquiring knowledge to derive modeling hypotheses. We provide an overview of these modeling principles and their application to transport systems, presenting numerous models that vary according to study objectives and corresponding modeling hypotheses. The data required for building, calibrating, and validating selected models are discussed, along with examples of using data analytics techniques to collect and handle the data supplied by ICT applications. The paper concludes with some comments on current and future trends