Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing

Within the context of autonomous driving a model-based reinforcement learning algorithm is proposed for the design of neural network-parameterized controllers. Classical model-based control methods, which include sampling- and lattice-based algorithms and model predictive control, suffer from the trade-off between model complexity and computational burden required for the online solution of expensive optimization or search problems at every short sampling time. To circumvent this trade-off, a 2-step procedure is motivated: first learning of a controller during offline training based on an arbitrarily complicated mathematical system model, before online fast feedforward evaluation of the trained controller. The contribution of this paper is the proposition of a simple gradient-free and model-based algorithm for deep reinforcement learning using task separation with hill climbing (TSHC). In particular, (i) simultaneous training on separate deterministic tasks with the purpose of encoding many motion primitives in a neural network, and (ii) the employment of maximally sparse rewards in combination with virtual velocity constraints (VVCs) in setpoint proximity are advocated.Comment: 10 pages, 6 figures, 1 tabl

Bidirectional Sampling Based Search Without Two Point Boundary Value Solution

Bidirectional motion planning approaches decrease planning time, on average, compared to their unidirectional counterparts. In single-query feasible motion planning, using bidirectional search to find a continuous motion plan requires an edge connection between the forward and reverse search trees. Such a tree-tree connection requires solving a two-point Boundary Value Problem (BVP). However, a two-point BVP solution can be difficult or impossible to calculate for many systems. We present a novel bidirectional search strategy that does not require solving the two-point BVP. Instead of connecting the forward and reverse trees directly, the reverse tree's cost information is used as a guiding heuristic for the forward search. This enables the forward search to quickly converge to a feasible solution without solving the two-point BVP. We propose two new algorithms (GBRRT and GABRRT) that use this strategy and run multiple software simulations using multiple dynamical systems and real-world hardware experiments to show that our algorithms perform on-par or better than existing state-of-the-art methods in quickly finding an initial feasible solution.Comment: Journal version (Video: https://youtu.be/Rumg66UHfyQ

Robotic Wireless Sensor Networks

In this chapter, we present a literature survey of an emerging, cutting-edge, and multi-disciplinary field of research at the intersection of Robotics and Wireless Sensor Networks (WSN) which we refer to as Robotic Wireless Sensor Networks (RWSN). We define a RWSN as an autonomous networked multi-robot system that aims to achieve certain sensing goals while meeting and maintaining certain communication performance requirements, through cooperative control, learning and adaptation. While both of the component areas, i.e., Robotics and WSN, are very well-known and well-explored, there exist a whole set of new opportunities and research directions at the intersection of these two fields which are relatively or even completely unexplored. One such example would be the use of a set of robotic routers to set up a temporary communication path between a sender and a receiver that uses the controlled mobility to the advantage of packet routing. We find that there exist only a limited number of articles to be directly categorized as RWSN related works whereas there exist a range of articles in the robotics and the WSN literature that are also relevant to this new field of research. To connect the dots, we first identify the core problems and research trends related to RWSN such as connectivity, localization, routing, and robust flow of information. Next, we classify the existing research on RWSN as well as the relevant state-of-the-arts from robotics and WSN community according to the problems and trends identified in the first step. Lastly, we analyze what is missing in the existing literature, and identify topics that require more research attention in the future

Advanced Modeling, Control, and Optimization Methods in Power Hybrid Systems - 2021

The climate changes that are becoming visible today are a challenge for the global research community. In this context, renewable energy sources, fuel cell systems and other energy generating sources must be optimally combined and connected to the grid system using advanced energy transaction methods. As this reprint presents the latest solutions in the implementation of fuel cell and renewable energy in mobile and stationary applications such as hybrid and microgrid power systems based on the Energy Internet, blockchain technology and smart contracts, we hope that they will be of interest to readers working in the related fields mentioned above

Machine Learning in Sensors and Imaging

Machine learning is extending its applications in various fields, such as image processing, the Internet of Things, user interface, big data, manufacturing, management, etc. As data are required to build machine learning networks, sensors are one of the most important technologies. In addition, machine learning networks can contribute to the improvement in sensor performance and the creation of new sensor applications. This Special Issue addresses all types of machine learning applications related to sensors and imaging. It covers computer vision-based control, activity recognition, fuzzy label classification, failure classification, motor temperature estimation, the camera calibration of intelligent vehicles, error detection, color prior model, compressive sensing, wildfire risk assessment, shelf auditing, forest-growing stem volume estimation, road management, image denoising, and touchscreens

Wide-Area Surveillance System using a UAV Helicopter Interceptor and Sensor Placement Planning Techniques

This project proposes and describes the implementation of a wide-area surveillance system comprised of a sensor/interceptor placement planning and an interceptor unmanned aerial vehicle (UAV) helicopter. Given the 2-D layout of an area, the planning system optimally places perimeter cameras based on maximum coverage and minimal cost. Part of this planning system includes the MATLAB implementation of Erdem and Sclaroff’s Radial Sweep algorithm for visibility polygon generation. Additionally, 2-D camera modeling is proposed for both fixed and PTZ cases. Finally, the interceptor is also placed to minimize shortest-path flight time to any point on the perimeter during a detection event. Secondly, a basic flight control system for the UAV helicopter is designed and implemented. The flight control system’s primary goal is to hover the helicopter in place when a human operator holds an automatic-flight switch. This system represents the first step in a complete waypoint-navigation flight control system. The flight control system is based on an inertial measurement unit (IMU) and a proportional-integral-derivative (PID) controller. This system is implemented using a general-purpose personal computer (GPPC) running Windows XP and other commercial off-the-shelf (COTS) hardware. This setup differs from other helicopter control systems which typically use custom embedded solutions or micro-controllers. Experiments demonstrate the sensor placement planning achieving \u3e90% coverage at optimized-cost for several typical areas given multiple camera types and parameters. Furthermore, the helicopter flight control system experiments achieve hovering success over short flight periods. However, the final conclusion is that the COTS IMU is insufficient for high-speed, high-frequency applications such as a helicopter control system

Measurements, Models, Systems and Design

531 s.

Low-Cost Multiple-MAV SLAM Using Open Source Software

We demonstrate a multiple micro aerial vehicle (MAV) system capable of supporting autonomous exploration and navigation in unknown environments using only a sensor commonly found in low-cost, commercially available MAVs—a front-facing monocular camera. We adapt a popular open source monocular SLAM library, ORB-SLAM, to support multiple inputs and present a system capable of effective cross-map alignment that can be theoretically generalized for use with other monocular SLAM libraries. Using our system, a single central ground control station is capable of supporting up to five MAVs simultaneously without a loss in mapping quality as compared to single-MAV ORB-SLAM. We conduct testing using both benchmark datasets and real-world trials to demonstrate the capability and real-time effectiveness

Microhydrodynamic, kinetic and thermal modeling of wet media milling for process optimization and intensification

Nanoparticle production by wet stirred media milling (WSMM) is a common method for the formulation of poorly water-soluble drugs. While most of the studies in the WSMM literature focus on the formulation aspects to overcome the stability challenges, a thorough mechanistic understanding of the process is lacking, and the process is slow, costly, and energy-intensive. This dissertation presents experimental and modeling work with the ultimate goals of (i) gaining a deeper and more mechanistic understanding of the WSMM process and breakage kinetics of the particles using a microhydrodynamic model with various improvements and advancements, (ii) examining the heat dissipation during the WSMM as a function of various process parameters, and (iii) optimizing and intensifying the WSMM using novel approaches such as bead mixtures of two different bead materials and mixtures of differently sized beads. To achieve the aforementioned goals, an nth-order breakage kinetics model is formulated to provide the best representation of the experimental median particle size evolution with time upon the milling of drug suspensions. Microhydrodynamic parameters are used to predict the breakage rate constant via a subset selection method, where the predictions are improved when the packing limit of the beads is taken into account. The analysis of heat generation–transfer experimental results suggest a significant rise in temperature during the milling, and stirrer speed is the most influential parameter followed by bead loading and bead size. An enthalpy balance model (EBM) is formulated to fit the experimental temperature profiles and determine the fraction of the mechanical power converted to heat, which is predicted using power law and machine learning approaches. As a low-fidelity alternative to the EBM, a semi-theoretical lumped-parameter model (LPM) is also formulated, which requires less experimental information though still provides a better estimation of temperature rise during WSMM as compared with the EBM. To improve the process, two novel process optimization approaches via bead mixtures are evaluated. When two bead materials, which are polystyrene and zirconia, are compared, polystyrene is found to be more efficient in terms of lower power consumption and heat generation, whereas zirconia beads are found to be better for fast breakage kinetics. Mixture of bead materials is introduced as a novel operational technique, to optimize the process from a holistic cycle time–power consumption–heat generation perspective. A decision tree for the composition of the bead mixture for various pharmaceutical application scenarios is developed. While the mixtures of polystyrene–zirconia beads help to reduce cycle time with acceptable temperature rise and power consumption, the mixtures of different bead sizes do not provide any significant benefit as compared with narrowly-sized individual beads. Overall, this dissertation addresses various process challenges of WSMM such as long cycle time and temperature rise, and formulates novel experimental solutions such as mixture of beads and predictive modeling techniques using various machine learning algorithms. Besides generating fundamental insights into the processing, the research hints at a new path to modeling the WSMM process via a combination of the microhydrodynamic model and population balance model augmented with machine learning approaches