Benchmarking Reinforcement Learning Techniques for Autonomous Navigation

Deep reinforcement learning (RL) has brought many successes for autonomous robot navigation. However, there still exists important limitations that prevent real-world use of RL-based navigation systems. For example, most learning approaches lack safety guarantees; and learned navigation systems may not generalize well to unseen environments. Despite a variety of recent learning techniques to tackle these challenges in general, a lack of an open-source benchmark and reproducible learning methods specifically for autonomous navigation makes it difficult for roboticists to choose what learning methods to use for their mobile robots and for learning researchers to identify current shortcomings of general learning methods for autonomous navigation. In this paper, we identify four major desiderata of applying deep RL approaches for autonomous navigation: (D1) reasoning under uncertainty, (D2) safety, (D3) learning from limited trial-and-error data, and (D4) generalization to diverse and novel environments. Then, we explore four major classes of learning techniques with the purpose of achieving one or more of the four desiderata: memory-based neural network architectures (D1), safe RL (D2), model-based RL (D2, D3), and domain randomization (D4). By deploying these learning techniques in a new open-source large-scale navigation benchmark and real-world environments, we perform a comprehensive study aimed at establishing to what extent can these techniques achieve these desiderata for RL-based navigation systems

Real-Time Hybrid Visual Servoing of a Redundant Manipulator via Deep Reinforcement Learning

Fixtureless assembly may be necessary in some manufacturing tasks and environ-ments due to various constraints but poses challenges for automation due to non-deterministic characteristics not favoured by traditional approaches to industrial au-tomation. Visual servoing methods of robotic control could be effective for sensitive manipulation tasks where the desired end-effector pose can be ascertained via visual cues. Visual data is complex and computationally expensive to process but deep reinforcement learning has shown promise for robotic control in vision-based manipu-lation tasks. However, these methods are rarely used in industry due to the resources and expertise required to develop application-specific systems and prohibitive train-ing costs. Training reinforcement learning models in simulated environments offers a number of benefits for the development of robust robotic control algorithms by reducing training time and costs, and providing repeatable benchmarks for which algorithms can be tested, developed and eventually deployed on real robotic control environments. In this work, we present a new simulated reinforcement learning envi-ronment for developing accurate robotic manipulation control systems in fixtureless environments. Our environment incorporates a contemporary collaborative industrial robot, the KUKA LBR iiwa, with the goal of positioning its end effector in a generic fixtureless environment based on a visual cue. Observational inputs are comprised of the robotic joint positions and velocities, as well as two cameras, whose positioning reflect hybrid visual servoing with one camera attached to the robotic end-effector, and another observing the workspace respectively. We propose a state-of-the-art deep reinforcement learning approach to solving the task environment and make prelimi-nary assessments of the efficacy of this approach to hybrid visual servoing methods for the defined problem environment. We also conduct a series of experiments ex-ploring the hyperparameter space in the proposed reinforcement learning method. Although we could not prove the efficacy of a deep reinforcement approach to solving the task environment with our initial results, we remain confident that such an ap-proach could be feasible to solving this industrial manufacturing challenge and that our contributions in this work in terms of the novel software provide a good basis for the exploration of reinforcement learning approaches to hybrid visual servoing in accurate manufacturing contexts

Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo

We introduce MuJoCo MPC (MJPC), an open-source, interactive application and software framework for real-time predictive control, based on MuJoCo physics. MJPC allows the user to easily author and solve complex robotics tasks, and currently supports three shooting-based planners: derivative-based iLQG and Gradient Descent, and a simple derivative-free method we call Predictive Sampling. Predictive Sampling was designed as an elementary baseline, mostly for its pedagogical value, but turned out to be surprisingly competitive with the more established algorithms. This work does not present algorithmic advances, and instead, prioritises performant algorithms, simple code, and accessibility of model-based methods via intuitive and interactive software. MJPC is available at: github.com/deepmind/mujoco_mpc, a video summary can be viewed at: dpmd.ai/mjpc.Comment: Minor fixes and formattin

Redistributed manufacturing in healthcare: Creating new value through disruptive innovation

The RiHN White Paper is the first serious attempt to gather expertise and to explore applications in promising areas of healthcare that could benefit from RDM and covers early-stage user needs, challenges and priorities. The UK has an opportunity to lead in this area and RiHN has identified an extensive number of areas for fruitful R&D, crossing production technology, infrastructure, business and organisations. The paper serves as a foundation for discussing future technological roadmaps and engaging the wider community and stakeholders, as well as policy makers, in addressing the potential impact of RDM.The RiHN White Paper is of particular value to policy makers and funders seeking to specify action and to direct attention where it is needed. The White Paper is also useful for the research community, to support their proposals with credible research propositions and to show where collaboration with industry and the public sector will deliver the most benefits.In order to seize the opportunities presented by RDM RiHN proposes a bold new agenda that incorporates a whole healthcare system view of future implementation pathways and wider transformation implications. The priority areas for Future R&D can be summarised as follows: throughAutomated production platform technologies and supporting manufacturing infrastructuresAdvances in analytics and metrologyNew regulatory frameworks and governance pathwaysNew frameworks for business model and organisational transformationThe time to take action is now. Technologies are developing that have the potential to disrupt traditional healthcare pathways and offer therapies tailored to individual needs and physiological characteristics. The challenge is seizing this opportunity and make the UK a world leader in RDM

Ethics, algorithms and self-driving cars – a CSI of the ‘trolley problem’. CEPS Policy Insights No 2018/02, January 2018

Many experts argue that focusing on how automated cars will solve the dilemma known as the ‘trolley problem’ isn’t going to get us very far in the debate about the ethics of artificial intelligence (AI). But it’s hard to resist if you are a philosopher, an ethicist, a futurist, or simply a geek – and it’s fun. Still, this dilemma can reveal a number of outstanding policy issues that are often neglected in the public debate. This paper performs a ‘crime scene investigation’ to find some of the missing parts in the ethics/AI quandary. These include the need to preserve human control over machines; the need to take data governance and ownership seriously; algorithmic accountability and transparency; various forms of user empowerment and their tension in relation to overall system control; the need for modernised tort rules; and more generally, a discussion about whether algorithms should reflect, exacerbate or mitigate the biases existing in our society. The investigation concludes that current legal systems are insufficiently equipped to cope with most of these issues, and that a mapping of outstanding ethical and policy dilemmas is a useful starting point for a thorough overhaul of public policies in this complex and ever-expanding domain

Analysis and evaluation of SafeDroid v2.0, a framework for detecting malicious Android applications

Android smartphones have become a vital component of the daily routine of millions of people, running a plethora of applications available in the official and alternative marketplaces. Although there are many security mechanisms to scan and filter malicious applications, malware is still able to reach the devices of many end-users. In this paper, we introduce the SafeDroid v2.0 framework, that is a flexible, robust, and versatile open-source solution for statically analysing Android applications, based on machine learning techniques. The main goal of our work, besides the automated production of fully sufficient prediction and classification models in terms of maximum accuracy scores and minimum negative errors, is to offer an out-of-the-box framework that can be employed by the Android security researchers to efficiently experiment to find effective solutions: the SafeDroid v2.0 framework makes it possible to test many different combinations of machine learning classifiers, with a high degree of freedom and flexibility in the choice of features to consider, such as dataset balance and dataset selection. The framework also provides a server, for generating experiment reports, and an Android application, for the verification of the produced models in real-life scenarios. An extensive campaign of experiments is also presented to show how it is possible to efficiently find competitive solutions: the results of our experiments confirm that SafeDroid v2.0 can reach very good performances, even with highly unbalanced dataset inputs and always with a very limited overhead