956 research outputs found
BackpropTools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control
Deep Reinforcement Learning (RL) has been demonstrated to yield capable
agents and control policies in several domains but is commonly plagued by
prohibitively long training times. Additionally, in the case of continuous
control problems, the applicability of learned policies on real-world embedded
devices is limited due to the lack of real-time guarantees and portability of
existing deep learning libraries. To address these challenges, we present
BackpropTools, a dependency-free, header-only, pure C++ library for deep
supervised and reinforcement learning. Leveraging the template meta-programming
capabilities of recent C++ standards, we provide composable components that can
be tightly integrated by the compiler. Its novel architecture allows
BackpropTools to be used seamlessly on a heterogeneous set of platforms, from
HPC clusters over workstations and laptops to smartphones, smartwatches, and
microcontrollers. Specifically, due to the tight integration of the RL
algorithms with simulation environments, BackpropTools can solve popular RL
problems like the Pendulum-v1 swing-up about 7 to 15 times faster in terms of
wall-clock training time compared to other popular RL frameworks when using
TD3. We also provide a low-overhead and parallelized interface to the MuJoCo
simulator, showing that our PPO implementation achieves state of the art
returns in the Ant-v4 environment while achieving a 25 to 30 percent faster
wall-clock training time. Finally, we also benchmark the policy inference on a
diverse set of microcontrollers and show that in most cases our optimized
inference implementation is much faster than even the manufacturer's DSP
libraries. To the best of our knowledge, BackpropTools enables the first-ever
demonstration of training a deep RL algorithm directly on a microcontroller,
giving rise to the field of Tiny Reinforcement Learning (TinyRL). Project page:
https://backprop.toolsComment: Project page: https://backprop.tool
Efficient Implementation of Complementary Golay Sequences for PAR Reduction and Forward Error Correction in OFDM-based WLAN systems
In this paper the use of complementary Golay sequences (CGS) for peak-to-average power ratio (PAR) reduction and forward error correction (FEC) in an orthogonal frequency division multiplexing (OFDM)-based wireless local area network (WLAN) system is explored; performance is examined and complexity issues are analyzed. We study their PAR reduction performance depending on sequence lengths and we have found that, for the case that the number of sub-carriers differs from the sequence length, some interesting relationships can still be stated. Regarding their error correction capabilities, these sequences are investigated considering M-PSK constellations applied to the OFDM signal specified in IEEE 802.11a standard. Computational load for both Golay encoding and decoding processes is addressed and we provide an exhaustive analysis of their complexity. In order to overcome memory restrictions and speed up algorithmic operations, a novel algorithm for real-time generation of the Golay Base Sequences is proposed and evaluated giving as a conclusion that these sequences can be real-time generated with actual Digital Signal Processors (DSP). Our proposal lies on an efficient permutation algorithm that obtains the current permutation without the need for generating previous ones. Its complexity is calculated and turns out to be significantly low; the advantages are specially appreciated at the decoding stage. We also introduce a hybrid solution to get a trade-off between complexity and memory requirements. Moreover, the whole system is also implemented in a DSP to validate the proposal in a prototype, where its feasibility has been confirmed.This work has been partly funded by the Spanish government with projects MACAWI (TEC 2005-07477-c02-02) and MAMBO (UC3M-TEC-05-027)
High-Precision Measurement of Sine and Pulse Reference Signals using Software-Defined Radio
This paper addresses simultaneous, high-precision measurement and analysis of
generic reference signals by using inexpensive commercial off-the-shelf
Software Defined Radio hardware. Sine reference signals are digitally
down-converted to baseband for the analysis of phase deviations. Hereby, we
compare the precision of the fixed-point hardware Digital Signal Processing
chain with a custom Single Instruction Multiple Data (SIMD) x86 floating-point
implementation. Pulse reference signals are analyzed by a software trigger that
precisely locates the time where the slope passes a certain threshold. The
measurement system is implemented and verified using the Universal Software
Radio Peripheral (USRP) N210 by Ettus Research LLC. Applying standard 10 MHz
and 1 PPS reference signals for testing, a measurement precision (standard
deviation) of 0.36 ps and 16.6 ps is obtained, respectively. In connection with
standard PC hardware, the system allows long-term acquisition and storage of
measurement data over several weeks. A comparison is given to the Dual Mixer
Time Difference (DMTD) and Time Interval Counter (TIC), which are
state-of-the-art measurement methods for sine and pulse signal analysis,
respectively. Furthermore, we show that our proposed USRP-based approach
outperforms measurements with a high-grade Digital Sampling Oscilloscope.Comment: 10 pages, 15 figures, and 4 table
- …