Combining Subgoal Graphs with Reinforcement Learning to Build a Rational Pathfinder

In this paper, we present a hierarchical path planning framework called SG-RL (subgoal graphs-reinforcement learning), to plan rational paths for agents maneuvering in continuous and uncertain environments. By "rational", we mean (1) efficient path planning to eliminate first-move lags; (2) collision-free and smooth for agents with kinematic constraints satisfied. SG-RL works in a two-level manner. At the first level, SG-RL uses a geometric path-planning method, i.e., Simple Subgoal Graphs (SSG), to efficiently find optimal abstract paths, also called subgoal sequences. At the second level, SG-RL uses an RL method, i.e., Least-Squares Policy Iteration (LSPI), to learn near-optimal motion-planning policies which can generate kinematically feasible and collision-free trajectories between adjacent subgoals. The first advantage of the proposed method is that SSG can solve the limitations of sparse reward and local minima trap for RL agents; thus, LSPI can be used to generate paths in complex environments. The second advantage is that, when the environment changes slightly (i.e., unexpected obstacles appearing), SG-RL does not need to reconstruct subgoal graphs and replan subgoal sequences using SSG, since LSPI can deal with uncertainties by exploiting its generalization ability to handle changes in environments. Simulation experiments in representative scenarios demonstrate that, compared with existing methods, SG-RL can work well on large-scale maps with relatively low action-switching frequencies and shorter path lengths, and SG-RL can deal with small changes in environments. We further demonstrate that the design of reward functions and the types of training environments are important factors for learning feasible policies.Comment: 20 page

RMPD - A Recursive Mid-Point Displacement Algorithm for Path Planning

Motivated by what is required for real-time path planning, the paper starts out by presenting sRMPD, a new recursive "local" planner founded on the key notion that, unless made necessary by an obstacle, there must be no deviation from the shortest path between any two points, which would normally be a straight line path in the configuration space. Subsequently, we increase the power of sRMPD by using it as a "connect" subroutine call in a higher-level sampling-based algorithm mRMPD that is inspired by multi-RRT. As a consequence, mRMPD spawns a larger number of space exploring trees in regions of the configuration space that are characterized by a higher density of obstacles. The overall effect is a hybrid tree growing strategy with a trade-off between random exploration as made possible by multi-RRT based logic and immediate exploitation of opportunities to connect two states as made possible by sRMPD. The mRMPD planner can be biased with regard to this trade-off for solving different kinds of planning problems efficiently. Based on the test cases we have run, our experiments show that mRMPD can reduce planning time by up to 80% compared to basic RRT

Articular human joint modelling

Copyright @ Cambridge University Press 2009.The work reported in this paper encapsulates the theories and algorithms developed to drive the core analysis modules of the software which has been developed to model a musculoskeletal structure of anatomic joints. Due to local bone surface and contact geometry based joint kinematics, newly developed algorithms make the proposed modeller different from currently available modellers. There are many modellers that are capable of modelling gross human body motion. Nevertheless, none of the available modellers offer complete elements of joint modelling. It appears that joint modelling is an extension of their core analysis capability, which, in every case, appears to be musculoskeletal motion dynamics. It is felt that an analysis framework that is focused on human joints would have significant benefit and potential to be used in many orthopaedic applications. The local mobility of joints has a significant influence in human motion analysis, in understanding of joint loading, tissue behaviour and contact forces. However, in order to develop a bone surface based joint modeller, there are a number of major problems, from tissue idealizations to surface geometry discretization and non-linear motion analysis. This paper presents the following: (a) The physical deformation of biological tissues as linear or non-linear viscoelastic deformation, based on spring-dashpot elements. (b) The linear dynamic multibody modelling, where the linear formulation is established for small motions and is particularly useful for calculating the equilibrium position of the joint. This model can also be used for finding small motion behaviour or loading under static conditions. It also has the potential of quantifying the joint laxity. (c) The non-linear dynamic multibody modelling, where a non-matrix and algorithmic formulation is presented. The approach allows handling complex material and geometrical nonlinearity easily. (d) Shortest path algorithms for calculating soft tissue line of action geometries. The developed algorithms are based on calculating minimum ‘surface mass’ and ‘surface covariance’. An improved version of the ‘surface covariance’ algorithm is described as ‘residual covariance’. The resulting path is used to establish the direction of forces and moments acting on joints. This information is needed for linear or non-linear treatment of the joint motion. (e) The final contribution of the paper is the treatment of the collision. In the virtual world, the difficulty in analysing bodies in motion arises due to body interpenetrations. The collision algorithm proposed in the paper involves finding the shortest projected ray from one body to the other. The projection of the body is determined by the resultant forces acting on it due to soft tissue connections under tension. This enables the calculation of collision condition of non-convex objects accurately. After the initial collision detection, the analysis involves attaching special springs (stiffness only normal to the surfaces) at the ‘potentially colliding points’ and motion of bodies is recalculated. The collision algorithm incorporates the rotation as well as translation. The algorithm continues until the joint equilibrium is achieved. Finally, the results obtained based on the software are compared with experimental results obtained using cadaveric joints

Multi UAV coverage path planning in urban environments

This article belongs to the Special Issue Efficient Planning and Mapping for Multi-Robot Systems.Coverage path planning (CPP) is a field of study which objective is to find a path that covers every point of a certain area of interest. Recently, the use of Unmanned Aerial Vehicles (UAVs) has become more proficient in various applications such as surveillance, terrain coverage, mapping, natural disaster tracking, transport, and others. The aim of this paper is to design efficient coverage path planning collision-avoidance capable algorithms for single or multi UAV systems in cluttered urban environments. Two algorithms are developed and explored: one of them plans paths to cover a target zone delimited by a given perimeter with predefined coverage height and bandwidth, using a boustrophedon flight pattern, while the other proposed algorithm follows a set of predefined viewpoints, calculating a smooth path that ensures that the UAVs pass over the objectives. Both algorithms have been developed for a scalable number of UAVs, which fly in a triangular deformable leader-follower formation with the leader at its front. In the case of an even number of UAVs, there is no leader at the front of the formation and a virtual leader is used to plan the paths of the followers. The presented algorithms also have collision avoidance capabilities, powered by the Fast Marching Square algorithm. These algorithms are tested in various simulated urban and cluttered environments, and they prove capable of providing safe and smooth paths for the UAV formation in urban environments.This research was funded by the EUROPEAN COMMISSION: Innovation and Networks Executive Agency (INEA), through the European H2020 LABYRINTH project. Grant agreement H2020-MG-2019-TwoStages-861696

Geometric-based Optimization Algorithms for Cable Routing and Branching in Cluttered Environments

The need for designing lighter and more compact systems often leaves limited space for planning routes for the connectors that enable interactions among the system’s components. Finding optimal routes for these connectors in a densely populated environment left behind at the detail design stage has been a challenging problem for decades. A variety of deterministic as well as heuristic methods has been developed to address different instances of this problem. While the focus of the deterministic methods is primarily on the optimality of the final solution, the heuristics offer acceptable solutions, especially for such problems, in a reasonable amount of time without guaranteeing to find optimal solutions. This study is an attempt to furthering the efforts in deterministic optimization methods to tackle the routing problem in two and three dimensions by focusing on the optimality of final solutions. The objective of this research is twofold. First, a mathematical framework is proposed for the optimization of the layout of wiring connectors in planar cluttered environments. The problem looks at finding the optimal tree network that spans multiple components to be connected with the aim of minimizing the overall length of the connectors while maximizing their common length (for maintainability and traceability of connectors). The optimization problem is formulated as a bi-objective problem and two solution methods are proposed: (1) to solve for the optimal locations of a known number of breakouts (where the connectors branch out) using mixed-binary optimization and visibility notion and (2) to find the minimum length tree that spans multiple components of the system and generates the optimal layout using the previously-developed convex hull based routing. The computational performance of these methods in solving a variety of problems is further evaluated. Second, the problem of finding the shortest route connecting two given nodes in a 3D cluttered environment is considered and addressed through deterministically generating a graphical representation of the collision-free space and searching for the shortest path on the found graph. The method is tested on sample workspaces with scattered convex polyhedra and its computational performance is evaluated. The work demonstrates the NP-hardness aspect of the problem which becomes quickly intractable as added components or increase in facets are considered

09111 Abstracts Collection -- Computational Geometry

From March 8 to March 13, 2009, the Dagstuhl Seminar 09111 ``Computational Geometry \u27\u27 was held in Schloss Dagstuhl~--~Leibniz Center for Informatics. During the seminar, several participants presented their current research, and ongoing work and open problems were discussed. Abstracts of the presentations given during the seminar as well as abstracts of seminar results and ideas are put together in this paper. The first section describes the seminar topics and goals in general. Links to extended abstracts or full papers are provided, if available

Smooth 3D Path Planning by Means of Multiobjective Optimization for Fixed-Wing UAVs

[EN] Demand for 3D planning and guidance algorithms is increasing due, in part, to the increase in unmanned vehicle-based applications. Traditionally, two-dimensional (2D) trajectory planning algorithms address the problem by using the approach of maintaining a constant altitude. Addressing the problem of path planning in a three-dimensional (3D) space implies more complex scenarios where maintaining altitude is not a valid approach. The work presented here implements an architecture for the generation of 3D flight paths for fixed-wing unmanned aerial vehicles (UAVs). The aim is to determine the feasible flight path by minimizing the turning effort, starting from a set of control points in 3D space, including the initial and final point. The trajectory generated takes into account the rotation and elevation constraints of the UAV. From the defined control points and the movement constraints of the UAV, a path is generated that combines the union of the control points by means of a set of rectilinear segments and spherical curves. However, this design methodology means that the problem does not have a single solution; in other words, there are infinite solutions for the generation of the final path. For this reason, a multiobjective optimization problem (MOP) is proposed with the aim of independently maximizing each of the turning radii of the path. Finally, to produce a complete results visualization of the MOP and the final 3D trajectory, the architecture was implemented in a simulation with Matlab/Simulink/flightGear.The authors would like to acknowledge the Spanish Ministerio de Ciencia, Innovacion y Universidades for providing funding through the project RTI2018-096904-B-I00 and the local administration Generalitat Valenciana through projects GV/2017/029 and AICO/2019/055. Franklin Samaniego thanks IFTH (Instituto de Fomento al Talento Humano) Ecuador (2015-AR2Q9209), for its sponsorship of this work.Samaniego, F.; Sanchís Saez, J.; Garcia-Nieto, S.; Simarro Fernández, R. (2020). Smooth 3D Path Planning by Means of Multiobjective Optimization for Fixed-Wing UAVs. Electronics. 9(1):1-23. https://doi.org/10.3390/electronics9010051S12391Kyriakidis, M., Happee, R., & de Winter, J. C. F. (2015). Public opinion on automated driving: Results of an international questionnaire among 5000 respondents. Transportation Research Part F: Traffic Psychology and Behaviour, 32, 127-140. doi:10.1016/j.trf.2015.04.014Münzer, S., Zimmer, H. D., Schwalm, M., Baus, J., & Aslan, I. (2006). Computer-assisted navigation and the acquisition of route and survey knowledge. Journal of Environmental Psychology, 26(4), 300-308. doi:10.1016/j.jenvp.2006.08.001Morales, Y., Kallakuri, N., Shinozawa, K., Miyashita, T., & Hagita, N. (2013). Human-comfortable navigation for an autonomous robotic wheelchair. 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. doi:10.1109/iros.2013.6696743Krotkov, E., & Hebert, M. (s. f.). Mapping and positioning for a prototype lunar rover. Proceedings of 1995 IEEE International Conference on Robotics and Automation. doi:10.1109/robot.1995.525697Rodriguez-Seda, E. J. (2014). Decentralized trajectory tracking with collision avoidance control for teams of unmanned vehicles with constant speed. 2014 American Control Conference. doi:10.1109/acc.2014.6859184Xiaoping Ren, & Zixing Cai. (2010). Kinematics model of unmanned driving vehicle. 2010 8th World Congress on Intelligent Control and Automation. doi:10.1109/wcica.2010.5554512Jun, J.-Y., Saut, J.-P., & Benamar, F. (2016). Pose estimation-based path planning for a tracked mobile robot traversing uneven terrains. Robotics and Autonomous Systems, 75, 325-339. doi:10.1016/j.robot.2015.09.014Li, Y., Ding, L., & Liu, G. (2016). Attitude-based dynamic and kinematic models for wheels of mobile robot on deformable slope. Robotics and Autonomous Systems, 75, 161-175. doi:10.1016/j.robot.2015.10.006Mekonnen, G., Kumar, S., & Pathak, P. M. (2016). Wireless hybrid visual servoing of omnidirectional wheeled mobile robots. Robotics and Autonomous Systems, 75, 450-462. doi:10.1016/j.robot.2015.08.008Xu, J., Wang, M., & Qiao, L. (2015). Dynamical sliding mode control for the trajectory tracking of underactuated unmanned underwater vehicles. Ocean Engineering, 105, 54-63. doi:10.1016/j.oceaneng.2015.06.022Gafurov, S. A., & Klochkov, E. V. (2015). Autonomous Unmanned Underwater Vehicles Development Tendencies. Procedia Engineering, 106, 141-148. doi:10.1016/j.proeng.2015.06.017Qi, X., & Cai, Z. (2018). Three-dimensional formation control based on nonlinear small gain method for multiple underactuated underwater vehicles. Ocean Engineering, 151, 105-114. doi:10.1016/j.oceaneng.2018.01.032Ramasamy, S., Sabatini, R., Gardi, A., & Liu, J. (2016). LIDAR obstacle warning and avoidance system for unmanned aerial vehicle sense-and-avoid. Aerospace Science and Technology, 55, 344-358. doi:10.1016/j.ast.2016.05.020Zhu, L., Cheng, X., & Yuan, F.-G. (2016). A 3D collision avoidance strategy for UAV with physical constraints. Measurement, 77, 40-49. doi:10.1016/j.measurement.2015.09.006Chee, K. Y., & Zhong, Z. W. (2013). Control, navigation and collision avoidance for an unmanned aerial vehicle. Sensors and Actuators A: Physical, 190, 66-76. doi:10.1016/j.sna.2012.11.017Courbon, J., Mezouar, Y., Guénard, N., & Martinet, P. (2010). Vision-based navigation of unmanned aerial vehicles. Control Engineering Practice, 18(7), 789-799. doi:10.1016/j.conengprac.2010.03.004Aguilar, W., & Morales, S. (2016). 3D Environment Mapping Using the Kinect V2 and Path Planning Based on RRT Algorithms. Electronics, 5(4), 70. doi:10.3390/electronics5040070Yan, F., Liu, Y.-S., & Xiao, J.-Z. (2013). Path Planning in Complex 3D Environments Using a Probabilistic Roadmap Method. International Journal of Automation and Computing, 10(6), 525-533. doi:10.1007/s11633-013-0750-9Yeh, H.-Y., Thomas, S., Eppstein, D., & Amato, N. M. (2012). UOBPRM: A uniformly distributed obstacle-based PRM. 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. doi:10.1109/iros.2012.6385875Liang, Y., & Xu, L. (2009). Global path planning for mobile robot based genetic algorithm and modified simulated annealing algorithm. Proceedings of the first ACM/SIGEVO Summit on Genetic and Evolutionary Computation - GEC ’09. doi:10.1145/1543834.1543875Liu, J., Yang, J., Liu, H., Tian, X., & Gao, M. (2016). An improved ant colony algorithm for robot path planning. Soft Computing, 21(19), 5829-5839. doi:10.1007/s00500-016-2161-7Cao, H., Sun, S., Zhang, K., & Tang, Z. (2016). Visualized trajectory planning of flexible redundant robotic arm using a novel hybrid algorithm. Optik, 127(20), 9974-9983. doi:10.1016/j.ijleo.2016.07.078Duan, H., & Qiao, P. (2014). Pigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning. International Journal of Intelligent Computing and Cybernetics, 7(1), 24-37. doi:10.1108/ijicc-02-2014-0005Pandey, A., & Parhi, D. R. (2017). Optimum path planning of mobile robot in unknown static and dynamic environments using Fuzzy-Wind Driven Optimization algorithm. Defence Technology, 13(1), 47-58. doi:10.1016/j.dt.2017.01.001Samaniego, F., Sanchis, J., García-Nieto, S., & Simarro, R. (2019). Recursive Rewarding Modified Adaptive Cell Decomposition (RR-MACD): A Dynamic Path Planning Algorithm for UAVs. Electronics, 8(3), 306. doi:10.3390/electronics8030306Dubins, L. E. (1957). On Curves of Minimal Length with a Constraint on Average Curvature, and with Prescribed Initial and Terminal Positions and Tangents. American Journal of Mathematics, 79(3), 497. doi:10.2307/2372560Fleury, S., Soueres, P., Laumond, J.-P., & Chatila, R. (1995). Primitives for smoothing mobile robot trajectories. IEEE Transactions on Robotics and Automation, 11(3), 441-448. doi:10.1109/70.388788Vanegas, G., Samaniego, F., Girbes, V., Armesto, L., & Garcia-Nieto, S. (2018). Smooth 3D path planning for non-holonomic UAVs. 2018 7th International Conference on Systems and Control (ICSC). doi:10.1109/icosc.2018.8587835Brezak, M., & Petrovic, I. (2014). Real-time Approximation of Clothoids With Bounded Error for Path Planning Applications. IEEE Transactions on Robotics, 30(2), 507-515. doi:10.1109/tro.2013.2283928Barsky, B. A., & DeRose, T. D. (1989). Geometric continuity of parametric curves: three equivalent characterizations. IEEE Computer Graphics and Applications, 9(6), 60-69. doi:10.1109/38.41470Kim, H., Kim, D., Shin, J.-U., Kim, H., & Myung, H. (2014). Angular rate-constrained path planning algorithm for unmanned surface vehicles. Ocean Engineering, 84, 37-44. doi:10.1016/j.oceaneng.2014.03.034Isaacs, J., & Hespanha, J. (2013). Dubins Traveling Salesman Problem with Neighborhoods: A Graph-Based Approach. Algorithms, 6(1), 84-99. doi:10.3390/a6010084Masehian, E., & Kakahaji, H. (2014). NRR: a nonholonomic random replanner for navigation of car-like robots in unknown environments. Robotica, 32(7), 1101-1123. doi:10.1017/s0263574713001276Fraichard, T., & Scheuer, A. (2004). From Reeds and Shepp’s to Continuous-Curvature Paths. IEEE Transactions on Robotics, 20(6), 1025-1035. doi:10.1109/tro.2004.833789Pepy, R., Lambert, A., & Mounier, H. (s. f.). Path Planning using a Dynamic Vehicle Model. 2006 2nd International Conference on Information & Communication Technologies. doi:10.1109/ictta.2006.1684472Girbés, V., Vanegas, G., & Armesto, L. (2019). Clothoid-Based Three-Dimensional Curve for Attitude Planning. Journal of Guidance, Control, and Dynamics, 42(8), 1886-1898. doi:10.2514/1.g003551De Lorenzis, L., Wriggers, P., & Hughes, T. J. R. (2014). Isogeometric contact: a review. GAMM-Mitteilungen, 37(1), 85-123. doi:10.1002/gamm.201410005Pigounakis, K. G., Sapidis, N. S., & Kaklis, P. D. (1996). Fairing Spatial B-Spline Curves. Journal of Ship Research, 40(04), 351-367. doi:10.5957/jsr.1996.40.4.351Pérez, L. H., Aguilar, M. C. M., Sánchez, N. M., & Montesinos, A. F. (2018). Path Planning Based on Parametric Curves. Advanced Path Planning for Mobile Entities. doi:10.5772/intechopen.72574Huh, U.-Y., & Chang, S.-R. (2014). A G2 Continuous Path-smoothing Algorithm Using Modified Quadratic Polynomial Interpolation. International Journal of Advanced Robotic Systems, 11(2), 25. doi:10.5772/57340Chang, S.-R., & Huh, U.-Y. (2014). A Collision-Free G2 Continuous Path-Smoothing Algorithm Using Quadratic Polynomial Interpolation. International Journal of Advanced Robotic Systems, 11(12), 194. doi:10.5772/59463Yaochu Jin, & Sendhoff, B. (2008). Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 38(3), 397-415. doi:10.1109/tsmcc.2008.919172Velasco-Carrau, J., García-Nieto, S., Salcedo, J. V., & Bishop, R. H. (2016). Multi-Objective Optimization for Wind Estimation and Aircraft Model Identification. Journal of Guidance, Control, and Dynamics, 39(2), 372-389. doi:10.2514/1.g001294Honig, E., Schucking, E. L., & Vishveshwara, C. V. (1974). Motion of charged particles in homogeneous electromagnetic fields. Journal of Mathematical Physics, 15(6), 774-781. doi:10.1063/1.1666728Iyer, B. R., & Vishveshwara, C. V. (1988). The Frenet-Serret formalism and black holes in higher dimensions. Classical and Quantum Gravity, 5(7), 961-970. doi:10.1088/0264-9381/5/7/005Laumanns, M., Thiele, L., Deb, K., & Zitzler, E. (2002). Combining Convergence and Diversity in Evolutionary Multiobjective Optimization. Evolutionary Computation, 10(3), 263-282. doi:10.1162/106365602760234108Blasco, X., Herrero, J. M., Sanchis, J., & Martínez, M. (2008). A new graphical visualization of n-dimensional Pareto front for decision-making in multiobjective optimization. Information Sciences, 178(20), 3908-3924. doi:10.1016/j.ins.2008.06.01