19 research outputs found
Air Learning: An AI Research Platform for Algorithm-Hardware Benchmarking of Autonomous Aerial Robots
We introduce Air Learning, an open-source simulator, and a gym environment
for deep reinforcement learning research on resource-constrained aerial robots.
Equipped with domain randomization, Air Learning exposes a UAV agent to a
diverse set of challenging scenarios. We seed the toolset with point-to-point
obstacle avoidance tasks in three different environments and Deep Q Networks
(DQN) and Proximal Policy Optimization (PPO) trainers. Air Learning assesses
the policies' performance under various quality-of-flight (QoF) metrics, such
as the energy consumed, endurance, and the average trajectory length, on
resource-constrained embedded platforms like a Raspberry Pi. We find that the
trajectories on an embedded Ras-Pi are vastly different from those predicted on
a high-end desktop system, resulting in up to longer trajectories in one
of the environments. To understand the source of such discrepancies, we use Air
Learning to artificially degrade high-end desktop performance to mimic what
happens on a low-end embedded system. We then propose a mitigation technique
that uses the hardware-in-the-loop to determine the latency distribution of
running the policy on the target platform (onboard compute on aerial robot). A
randomly sampled latency from the latency distribution is then added as an
artificial delay within the training loop. Training the policy with artificial
delays allows us to minimize the hardware gap (discrepancy in the flight time
metric reduced from 37.73\% to 0.5\%). Thus, Air Learning with
hardware-in-the-loop characterizes those differences and exposes how the
onboard compute's choice affects the aerial robot's performance. We also
conduct reliability studies to assess the effect of sensor failures on the
learned policies. All put together, \airl enables a broad class of deep RL
research on UAVs. The source code is available
at:~\texttt{\url{http://bit.ly/2JNAVb6}}.Comment: To Appear in Springer Machine Learning Journal (Special Issue on
Reinforcement Learning for Real Life
Widening Access to Applied Machine Learning with TinyML
Broadening access to both computational and educational resources is critical
to diffusing machine-learning (ML) innovation. However, today, most ML
resources and experts are siloed in a few countries and organizations. In this
paper, we describe our pedagogical approach to increasing access to applied ML
through a massive open online course (MOOC) on Tiny Machine Learning (TinyML).
We suggest that TinyML, ML on resource-constrained embedded devices, is an
attractive means to widen access because TinyML both leverages low-cost and
globally accessible hardware, and encourages the development of complete,
self-contained applications, from data collection to deployment. To this end, a
collaboration between academia (Harvard University) and industry (Google)
produced a four-part MOOC that provides application-oriented instruction on how
to develop solutions using TinyML. The series is openly available on the edX
MOOC platform, has no prerequisites beyond basic programming, and is designed
for learners from a global variety of backgrounds. It introduces pupils to
real-world applications, ML algorithms, data-set engineering, and the ethical
considerations of these technologies via hands-on programming and deployment of
TinyML applications in both the cloud and their own microcontrollers. To
facilitate continued learning, community building, and collaboration beyond the
courses, we launched a standalone website, a forum, a chat, and an optional
course-project competition. We also released the course materials publicly,
hoping they will inspire the next generation of ML practitioners and educators
and further broaden access to cutting-edge ML technologies.Comment: Understanding the underpinnings of the TinyML edX course series:
https://www.edx.org/professional-certificate/harvardx-tiny-machine-learnin
Widening Access to Applied Machine Learning With TinyML
Broadening access to both computational and educational resources is crit- ical to diffusing machine learning (ML) innovation. However, today, most ML resources and experts are siloed in a few countries and organizations. In this article, we describe our pedagogical approach to increasing access to applied ML through a massive open online course (MOOC) on Tiny Machine Learning (TinyML). We suggest that TinyML, applied ML on resource-constrained embedded devices, is an attractive means to widen access because TinyML leverages low-cost and globally accessible hardware and encourages the development of complete, self-contained applications, from data collection to deployment. To this end, a collaboration between academia and industry produced a four part MOOC that provides application-oriented instruction on how to develop solutions using TinyML. The series is openly available on the edX MOOC platform, has no prerequisites beyond basic programming, and is designed for global learners from a variety of backgrounds. It introduces real-world applications, ML algorithms, data-set engineering, and the ethi- cal considerations of these technologies through hands-on programming and deployment of TinyML applications in both the cloud and on their own microcontrollers. To facili- tate continued learning, community building, and collaboration beyond the courses, we launched a standalone website, a forum, a chat, and an optional course-project com- petition. We also open-sourced the course materials, hoping they will inspire the next generation of ML practitioners and educators and further broaden access to cutting-edge ML technologies
QuaRL: Quantization for Sustainable Reinforcement Learning
Deep reinforcement learning has achieved significant milestones, however, the
computational demands of reinforcement learning training and inference remain
substantial. Quantization is an effective method to reduce the computational
overheads of neural networks, though in the context of reinforcement learning,
it is unknown whether quantization's computational benefits outweigh the
accuracy costs introduced by the corresponding quantization error. To quantify
this tradeoff we perform a broad study applying quantization to reinforcement
learning. We apply standard quantization techniques such as post-training
quantization (PTQ) and quantization aware training (QAT) to a comprehensive set
of reinforcement learning tasks (Atari, Gym), algorithms (A2C, DDPG, DQN, D4PG,
PPO), and models (MLPs, CNNs) and show that policies may be quantized to 8-bits
without degrading reward, enabling significant inference speedups on
resource-constrained edge devices. Motivated by the effectiveness of standard
quantization techniques on reinforcement learning policies, we introduce a
novel quantization algorithm, \textit{ActorQ}, for quantized actor-learner
distributed reinforcement learning training. By leveraging full precision
optimization on the learner and quantized execution on the actors,
\textit{ActorQ} enables 8-bit inference while maintaining convergence. We
develop a system for quantized reinforcement learning training around
\textit{ActorQ} and demonstrate end to end speedups of 1.5 - 2.5
over full precision training on a range of tasks (Deepmind Control
Suite). Finally, we break down the various runtime costs of distributed
reinforcement learning training (such as communication time, inference time,
model load time, etc) and evaluate the effects of quantization on these system
attributes.Comment: Equal contribution from first three authors. Updating with QuaRL for
sustainable (carbon emissions) RL result
Effect of friction stir welding on microstructure, tensile and fatigue properties of the AA7005/10 vol.%Al2O3 composite
Tiny Robot Learning: Challenges and Directions for Machine Learning in Resource-Constrained Robots
Machine learning (ML) has become a pervasive tool across computing systems. An emerging application that stress-tests the challenges of ML system design is tiny robot learning, the deployment of ML on resource-constrained low-cost autonomous robots. Tiny robot learning lies at the intersection of embedded systems, robotics, and ML, compounding the challenges of these domains. Tiny robot learning is subject to challenges from size, weight, area, and power (SWAP) constraints; sensor, actuator, and compute hardware limitations; end-to-end system tradeoffs; and a large diversity of possible deployment scenarios. Tiny robot learning requires ML models to be designed with these challenges in mind, providing a crucible that reveals the necessity of holistic ML system design and automated end-to-end design tools for agile development. This paper gives a brief survey of the tiny robot learning space, elaborates on key challenges, and proposes promising opportunities for future work in ML system design.Green Open Access added to TU Delft Institutional Repository ‘You share, we take care!’ – Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.Control & Simulatio