5 research outputs found
The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Cyber-Physical Systems
This paper analyzes the simulation to reality gap in reinforcement learning
(RL) cyber-physical systems with fractional delays (i.e. delays that are
non-integer multiple of the sampling period). The consideration of fractional
delay has important implications on the nature of the cyber-physical system
considered. Systems with delays are non-Markovian, and the system state vector
needs to be extended to make the system Markovian. We show that this is not
possible when the delay is in the output, and the problem would always be
non-Markovian. Based on this analysis, a sampling scheme is proposed that
results in efficient RL training and agents that perform well in realistic
multirotor unmanned aerial vehicle simulations. We demonstrate that the
resultant agents do not produce excessive oscillations, which is not the case
with RL agents that do not consider time delay in the model.Comment: 6 pages,4 figures, Submitted to ICRA202
Real-time system identification using deep learning for linear processes with application to unmanned aerial vehicles
This paper proposes a novel parametric identification approach for linear
systems using Deep Learning (DL) and the Modified Relay Feedback Test (MRFT).
The proposed methodology utilizes MRFT to reveal distinguishing frequencies
about an unknown process; which are then passed to a trained DL model to
identify the underlying process parameters. The presented approach guarantees
stability and performance in the identification and control phases
respectively, and requires few seconds of observation data to infer the dynamic
system parameters. Quadrotor Unmanned Aerial Vehicle (UAV) attitude and
altitude dynamics were used in simulation and experimentation to verify the
presented methodology. Results show the effectiveness and real-time
capabilities of the proposed approach, which outperforms the conventional
Prediction Error Method in terms of accuracy, robustness to biases,
computational efficiency and data requirements.Comment: 13 pages, 9 figures. Submitted to IEEE access. A supplementary video
for the work presented in this paper can be accessed at:
https://www.youtube.com/watch?v=dz3WTFU7W7c. This version includes minor
style edits for appendix and reference
Design of Dynamics Invariant LSTM for Touch Based Human-UAV Interaction Detection
The field of Unmanned Aerial Vehicles (UAVs) has reached a high level of
maturity in the last few years. Hence, bringing such platforms from closed
labs, to day-to-day interactions with humans is important for commercialization
of UAVs. One particular human-UAV scenario of interest for this paper is the
payload handover scheme, where a UAV hands over a payload to a human upon their
request. In this scope, this paper presents a novel real-time human-UAV
interaction detection approach, where Long short-term memory (LSTM) based
neural network is developed to detect state profiles resulting from human
interaction dynamics. A novel data pre-processing technique is presented; this
technique leverages estimated process parameters of training and testing UAVs
to build dynamics invariant testing data. The proposed detection algorithm is
lightweight and thus can be deployed in real-time using off the shelf UAV
platforms; in addition, it depends solely on inertial and position measurements
present on any classical UAV platform. The proposed approach is demonstrated on
a payload handover task between multirotor UAVs and humans. Training and
testing data were collected using real-time experiments. The detection approach
has achieved an accuracy of 96\%, giving no false positives even in the
presence of external wind disturbances, and when deployed and tested on two
different UAVs.Comment: 13 pages, 13 figures, submitted to IEEE access, A supplementary video
for the work presented in this paper can be accessed from
https://youtu.be/29N_OXBl1m