The Role of Time Delay in Sim2real Transfer of Reinforcement Learning for Cyber-Physical Systems

This paper analyzes the simulation to reality gap in reinforcement learning (RL) cyber-physical systems with fractional delays (i.e. delays that are non-integer multiple of the sampling period). The consideration of fractional delay has important implications on the nature of the cyber-physical system considered. Systems with delays are non-Markovian, and the system state vector needs to be extended to make the system Markovian. We show that this is not possible when the delay is in the output, and the problem would always be non-Markovian. Based on this analysis, a sampling scheme is proposed that results in efficient RL training and agents that perform well in realistic multirotor unmanned aerial vehicle simulations. We demonstrate that the resultant agents do not produce excessive oscillations, which is not the case with RL agents that do not consider time delay in the model.Comment: 6 pages,4 figures, Submitted to ICRA202

Methodology for the Development of Radar Control Systems for Flying Targets with an Artificially Reduced RCS

The article explores methods for detecting and tracking air targets in radar with an artificially reduced radar cross-section (RCS). A technique for control the radar antenna system using an adaptive method is presented, which is based on adapting the system to the conditions of information uncertainty due to fluctuations in the excitation signal of the radar target's radar-absorbing coating. Known methods of radar aircraft (active and passive, which do not take into account the structure of the coating of the aircraft) do not allow obtaining information about an air target with an artificially reduced RCS. The main contribution of this article is the development of radar and control methods based on the resonant frequency-phase interaction of the microwave electromagnetic field with the crystal structure of the radar absorbing coating of the aircraft. The stages of antenna system control based on the "frequency-phase detection method", "passive-active" tracking method, and "adaptive method" of antenna control have been studied. To test the proposed methods, an experiment was conducted to determine the transient process in the control system under conditions of information uncertainty. As a result of the experiment, the probability of tracking the target is increased by 14-19%. The findings will be useful for developers of radar and control systems for modern air facilities with an artificially reduced RCS

Engineering Practices of Determining Transmission Capacity and Delay of Interconnecting Line Taking into Account its Configuration and Cost

This article contains information on engineering practice of determining transmission capacity of computer network line. The article presents a variant of engineering synthesis of computer network, which is a combined process of mathematical and heuristic methods combining. The engineering synthesis is offered as vector and global, because it must result in network development, optimal in terms of its practical use. All the significant network quality indicators, including economic and practical, are taken into consideration. In case of engineering synthesis, it is not possible that only one quality indicator is significant: there are always at least two significant indicators – a cost and an indicator that characterizes the main effect that is achieved in case of network use (efficacy). If at least one of the quality indicators significant for practical use is not taken into account, such network cannot be considered optimal. Computer network synthesis usually consists of structure synthesis, parameters optimization and discrete network selection. If network topology is maintained unchanged, it is possible to formulate an optimization task for line transmission capacity. The solution of transmission capacity task, which is constantly changing, may be chosen as a starting point for the selection of discrete indicator of transmission capacity

Experimental Studies on the Reactive Thrust of the Mobile Robot of Arbitrary Orientation

The problem of creating mobile robots of arbitrary orientation in the technological space is to ensure reliable retention of robots on the surface of any orientation. Therefore, well-known experimental studies are mainly devoted to the creation of systems for coupling the robot to the surface along which it moves. The purpose of this study is to create a device for compensating the gravitational load of a mobile robot. The article contains the results of experimental testing of a fundamentally new approach to counteract the gravitational load of a mobile robot, namely, the expediency of equipping the robot with a source of reactive thrust of a non-chemical origin. A pneumatic generator of aerodynamic lift is proposed as such a source. Such a force partially compensates or completely overcomes the gravitational load, while not allowing the transformation of a mobile robot into an aircraft. The specified condition is necessary to perform contact power technological operations in the maintenance of various industrial facilities. In other words, the thrust force should not exceed the adhesion forces of the mobile robot to the displacement surface. As a research method, a full factorial experiment of the operation of a jet thrust generator was used, which is a new way to increase the reliability of holding the robot on an arbitrary surface. The article describes the methodology and description of the full factorial experiment with varying independent factors at two extreme levels. As a result, an experimental solution to the problem of finding the quasi-optimal values of the aerodynamic lifting force depending on the parameters of the jet thrust generator is obtained. As a result, the combination of a new robot design with the results of experimental studies confirms the feasibility of using a pneumatic jet thrust generator as a means of increasing the reliability of holding mobile robots on an arbitrary orientation surface in the technological space

Assessment of quality indicators of the automatic control system influence of accident interference

This work concentrates the analysis of the system of automatic control of the directive diagram of the moving active electronically scanned array with a limited number of transceiver modules. The analysis revealed a number of shortcomings that lead to a significant increase in standard deviations, quadratic integral estimates, and an increase in transient time. The identified disadvantages lead to a decrease in the efficiency of the antenna system, an increase in the error rate at the reception, the inability of the system to react to disturbances applied to any point of the system in the event of a mismatch of a given signal/noise level. In accordance with the analysis, the mathematical model of the automatic control system of the directional diagram of the moving active electronically scanned array was considered, considering this a new method of estimating the quality indicators of the automatic control diagram of the directional diagram of the active electronically scanned array in a random setting and disturbing action was developed. The difference between the proposed method and the existing method is in the construction of an automatic control system with differential coupling equivalent to the combination due to the introduction of derivatives of the random setting action of the open compensation connection

Fuzzy Ensembles of Reinforcement Learning Policies for Robotic Systems with Varied Parameters

Reinforcement Learning (RL) is an emerging approach to control many dynamical systems for which classical control approaches are not applicable or insufficient. However, the resultant policies may not generalize to variations in the parameters that the system may exhibit. This paper presents a powerful yet simple algorithm in which collaboration is facilitated between RL agents that are trained independently to perform the same task but with different system parameters. The independency among agents allows the exploitation of multi-core processing to perform parallel training. Two examples are provided to demonstrate the effectiveness of the proposed technique. The main demonstration is performed on a quadrotor with slung load tracking problem in a real-time experimental setup. It is shown that integrating the developed algorithm outperforms individual policies by reducing the RMSE tracking error. The robustness of the ensemble is also verified against wind disturbance.Comment: arXiv admin note: text overlap with arXiv:2311.0501

MRFT-based design of robust and adaptive controllers for gas loop of oil-gas separator

Abstract: The modified relay feedback test (MRFT), which was recently proposed as a continuous oscillation method for identification of the process parameters and controller tuning, is used for the design of a robust and an adaptive ProportionalIntegral (PI) controller for a gas loop in the oil-gas separator. The gas normally found in the separator is the natural gas (mostly methane) which is contained in crude oil coming from the reservoir. The robust and adaptive PI controllers are developed from analysis of 64 operating modes corresponding to certain ranges of the gas inflow and liquid-level values. It is shown through the developed model and simulations that these operating modes have significant effect on the dynamics of the gas loop. Dynamic properties of the process in each mode are studied through MRFT. The controllers are designed in order to maintain the pressure during the change of the operating conditions. Performance of the designed control system is studied by simulations

Evaluation of Load Bearing Capacity of Foundations with Different Vertical Cross- sectional Shapes

ABSTRACT: The paper presents results of experimental and theoreticalevaluation of load bearing capacity of shallow foundations with different vertical cross-sectional shapes. Load-settlement relationships ofmodels of shallow foundations with rectangular, wedge and T shape vertical cross-sections were studied through experimentation, and theoretically using Terzaghi bearing capacity equation.The experimental and theoretical evaluations show the load bearing capacity increasing in the order: T shape > wedge shape >rectangular shape. The study indicate that the soil above the bases of shallow foundations with wedge and T vertical cross-sectional shapes, function not only as surcharge to the soil below the bases (as is the case of rectangular shape), but also offers additional resistance to structural loads, and thereby increasing the load bearing capacity of the foundations. The economicbenefit of using wedge and T shapes shallow foundations is also demonstrated

Design of Dynamics Invariant LSTM for Touch Based Human-UAV Interaction Detection

The field of Unmanned Aerial Vehicles (UAVs) has reached a high level of maturity in the last few years. Hence, bringing such platforms from closed labs, to day-to-day interactions with humans is important for commercialization of UAVs. One particular human-UAV scenario of interest for this paper is the payload handover scheme, where a UAV hands over a payload to a human upon their request. In this scope, this paper presents a novel real-time human-UAV interaction detection approach, where Long short-term memory (LSTM) based neural network is developed to detect state profiles resulting from human interaction dynamics. A novel data pre-processing technique is presented; this technique leverages estimated process parameters of training and testing UAVs to build dynamics invariant testing data. The proposed detection algorithm is lightweight and thus can be deployed in real-time using off the shelf UAV platforms; in addition, it depends solely on inertial and position measurements present on any classical UAV platform. The proposed approach is demonstrated on a payload handover task between multirotor UAVs and humans. Training and testing data were collected using real-time experiments. The detection approach has achieved an accuracy of 96\%, giving no false positives even in the presence of external wind disturbances, and when deployed and tested on two different UAVs.Comment: 13 pages, 13 figures, submitted to IEEE access, A supplementary video for the work presented in this paper can be accessed from https://youtu.be/29N_OXBl1m