Evolutionary algorithms for robot path planning, task allocation and collision avoidance in an automated warehouse

Thesis (PhD)--Stellenbosch University, 2022.ENGLISH ABSTRACT: Research with regard to path planning, task allocation and collision avoidance is important for improving the field of warehouse automation. The dissertation addresses the topic of routing warehouse picking and binning robots. The purpose of this dissertation is to develop a single objective and multi-objective algorithm framework that can sequence products to be picked or binned, allocate the products to robots and optimise the routing through the warehouse. The sequence of the picking and binning tasks ultimately determines the total time for picking and binning all of the parts. The objectives of the algorithm framework are to minimise the total time for travelling as well as the total time idling, given the number of robots available to perform the picking and binning functions. The algorithm framework incorporates collision avoidance since the aisle width does not allow two robots to pass each other. The routing problem sets the foundation for solving the sequencing and allocation problem. The best heuristic from the routing problem is used as the strategy for routing the robots in the sequencing and allocation problem. The routing heuristics used to test the framework in this dissertation include the return heuristic, the s-shape heuristic, the midpoint heuristic and the largest gap heuristic. The metaheuristic solution strategies for single objective part sequencing and allocating problem include the covariance matrix adaptation evolution strategy (CMA-ES) algorithm, the genetic algorithm (GA), the guaranteed convergence particle swarm optimisation (GCPSO) algorithm, and the self-adaptive differential evolution algorithm with neighbourhood search (SaNSDE). The evolutionary multi-objective algorithms considered in this dissertation are the non-dominated sorting genetic algorithm III (NSGA-III), the multi-objective evolutionary algorithm based on decomposition (MOEAD), the multiple objective particle swarm optimisation (MOPSO), and the multi-objective covariance matrix adaptation evolution strategy (MO-CMA-ES). Solving the robot routing problem showed that the return routing heuristic outperformed the s-shape, largest gap and midpoint heuristics with a significant margin. The return heuristic was thus used for solving the routing of robots in the part sequencing and allocation problem. The framework was able to create feasible real-world solutions for the part sequencing and allocation problem. The results from the single objective problem showed that the CMA-ES algorithm outperformed the other metaheuristics on the part sequencing and allocation problem. The second best performing metaheuristic was the SaNSDE. The GA was the third best metaheuristic and the worst performing metaheuristic was the GCPSO. The multi-objective framework was able to produce feasible trade-off solutions and MOPSO was shown to be the best EMO algorithm to use for accuracy. If a large spread and number of Pareto solutions are the most important concern, MOEAD should be used. The research contributions include the incorporation of collision avoidance in the robot routing problem when using single and multi-objective algorithms as solution strategies. This dissertation contributes to the research relating to the performance of metaheuristics and evolutionary multi-objective algorithms on routing, sequencing, and allocation problems. To the best of the author’s knowledge, this dissertation is the first where these four metaheuristics and evolutionary multi-objective algorithms have been tested for solving the robot picking and binning problem, given that all collisions must be avoided. It is also the first time that this specific variation of the part sequencing and allocation problem has been solved using metaheuristics and evolutionary multi-objective algorithms, taking into account that all collisions must be avoided.AFRIKAANSE OPSOMMING: Navorsing in verband met roete beplanning, part allokasie en botsing vermyding is belangrik vir die bevordering van die pakhuis automatisering veld. Die verhandeling handel oor die onderwerp van parte wat gestoor en gehaal moet word en die verkillende parte moet ook gealokeer word aan ’n spesifieke robot. Die doel van hierdie verhandeling is om ’n enkele doelwit en ’n multidoelwit algoritme raamwerk te ontwikkel wat parte in ’n volgorde rangskik en ook die parte aan ’n robot alokeer. Die roete wat die robot moet volg deur die pakhuis moet ook geoptimeer word om die minste tyd te verg. Die volgorde van die parte bepaal uiteindelik die totale tyd wat dit neem vir die robot om al die parte te stoor en te gaan haal. Die doelwitte van die algoritme raamwerk is om die totale reistyd en die totale ledige tyd te minimeer, gegewe die aantal beskikbare robotte in die sisteem om die stoor en gaan haal funksies uit te voer. Die algoritme raamwerk bevat botsingsvermyding, aangesien die gangbreedte van die pakhuis nie toelaat dat twee robotte mekaar kan verbygaan nie. Die roete probleem lˆe die grondslag vir die oplossing van die volgorde en allokerings probleem. Die beste heuristiek vir die roete probleem word verder gebruik in die volgorde en allokerings probleem. Die verskillende roete heuristieke wat in hierdie verhandeling oorweeg was, sluit in die terugkeer heuristiek, die s-vorm heuristiek, die middelpunt heuristiek en die grootste gaping heuristiek. Die metaheuristieke vir die volgorde en allokerings probleem sluit die volgende algoritmes in: die kovariansie matriks aanpassing evolusie algoritme (CMA-ES), die genetiese algoritme (GA), die gewaarborgde konvergerende deeltjie swermoptimerings (GCPSO) algoritme, en laastens die selfaanpassende differensi¨ele evolusie algoritme met die teenwoordigheid van buurtsoek (SaNSDE). Die evolusionêre multidoelwit algoritmes wat oorweeg was vir die volgorde en allokerings probleem sluit die volgende algoritmes in: die multidoelwit kovariansie matriks aanpassing evolusie algoritme (MO-CMA-ES), die nie-dominerende sortering genetiese algoritme III (NSGA-III), die multidoelwit evolusionˆere algoritme gebaseer op ontbinding (MOEAD) en laastens die multidoelwit deeltjie swermoptimering algoritme (MOPSO) Oplossings van die robot roete probleem het gewys dat die terugkeer heuristiek die s-vorm, grootste gaping en middelpunt heuristiek met ’n beduidende marge oortref het. Die terugkeer heuristiek is dus gebruik vir die oplossing van die roete beplanning van robotte in die volgorde en allokasie probleem. Die raamwerk was doeltreffend en die resultate het getoon, vir die enkel doelwit probleem, dat die CMA-ES algoritme beter gevaar het as die ander metaheuristieke vir die volgorde en allokasie probleem. Die SaNSDE was die naas beste presterende metaheuristiek. Die GA was die derde beste metaheuristiek, en die metaheuristiek wat die slegste gevaar het, was die GCPSO. Vir die multidoelwit probleem het die MOPSO die beste gevaar, as akkuraatheid die belangrikste doelwit is. As ’n grootter verskeidenheid die belangrikste is, is die MOEAD meer geskik om ’n oplossing te vind. Die navorsingsbydraes sluit in dat vermyding van botsings in ag geneem word in die robot roete probleem. Hierdie verhandeling dra by tot die navorsing oor die oplossing van roete beplanning, volgorde en allokasie probleme met metaheuristieke. Na die beste van die outeur se kennis is hierdie die eerste keer dat al vier metaheuristieke getoets was om die robot stoor-en-gaan haal probleem op te los, onder die kondisie dat alle botsings vermy moet word. Dit is ook die eerste keer dat hierdie spesifieke variant, enkel-en-multidoelwit probleem van die volgorde en allokasie van parte met behulp van metaheuristieke en multidoelwit evolusionˆere algoritmes opgelos was, met die inagneming dat alle botsings vermy moet word.Doctora

An Approach to Improve Multi objective Path Planning for Mobile Robot Navigation using the Novel Quadrant Selection Method

Currently, automated and semi-automated industries need multiple objective path planning algorithms for mobile robot applications. The multi-objective optimisation algorithm takes more computational effort to provide optimal solutions. The proposed grid-based multi-objective global path planning algorithm [Quadrant selection algorithm (QSA)] plans the path by considering the direction of movements from starting position to the target position with minimum computational effort. Primarily, in this algorithm, the direction of movements is classified into quadrants. Based on the selection of the quadrant, the optimal paths are identified. In obstacle avoidance, the generated feasible paths are evaluated by the cumulative path distance travelled, and the cumulative angle turned to attain an optimal path. Finally, to ease the robot’s navigation, the obtained optimal path is further smoothed to avoid sharp turns and reduce the distance. The proposed QSA in total reduces the unnecessary search for paths in other quadrants. The developed algorithm is tested in different environments and compared with the existing algorithms based on the number of cells examined to obtain the optimal path. Unlike other algorithms, the proposed QSA provides an optimal path by dramatically reducing the number of cells examined. The experimental verification of the proposed QSA shows that the solution is practically implementable

Agriculture fleet vehicle routing: A decentralised and dynamic problem

To date, the research on agriculture vehicles in general and Agriculture Mobile Robots (AMRs) in particular has focused on a single vehicle (robot) and its agriculture-specific capabilities. Very little work has explored the coordination of fleets of such vehicles in the daily execution of farming tasks. This is especially the case when considering overall fleet performance, its efficiency and scalability in the context of highly automated agriculture vehicles that perform tasks throughout multiple fields potentially owned by different farmers and/or enterprises. The potential impact of automating AMR fleet coordination on commercial agriculture is immense. Major conglomerates with large and heterogeneous fleets of agriculture vehicles could operate on huge land areas without human operators to effect precision farming. In this paper, we propose the Agriculture Fleet Vehicle Routing Problem (AF-VRP) which, to the best of our knowledge, differs from any other version of the Vehicle Routing Problem studied so far. We focus on the dynamic and decentralised version of this problem applicable in environments involving multiple agriculture machinery and farm owners where concepts of fairness and equity must be considered. Such a problem combines three related problems: the dynamic assignment problem, the dynamic 3-index assignment problem and the capacitated arc routing problem. We review the state-of-the-art and categorise solution approaches as centralised, distributed and decentralised, based on the underlining decision-making context. Finally, we discuss open challenges in applying distributed and decentralised coordination approaches to this problem

Best matching processes in distributed systems

The growing complexity and dynamic behavior of modern manufacturing and service industries along with competitive and globalized markets have gradually transformed traditional centralized systems into distributed networks of e- (electronic) Systems. Emerging examples include e-Factories, virtual enterprises, smart farms, automated warehouses, and intelligent transportation systems. These (and similar) distributed systems, regardless of context and application, have a property in common: They all involve certain types of interactions (collaborative, competitive, or both) among their distributed individuals—from clusters of passive sensors and machines to complex networks of computers, intelligent robots, humans, and enterprises. Having this common property, such systems may encounter common challenges in terms of suboptimal interactions and thus poor performance, caused by potential mismatch between individuals. For example, mismatched subassembly parts, vehicles—routes, suppliers—retailers, employees—departments, and products—automated guided vehicles—storage locations may lead to low-quality products, congested roads, unstable supply networks, conflicts, and low service level, respectively. This research refers to this problem as best matching, and investigates it as a major design principle of CCT, the Collaborative Control Theory. The original contribution of this research is to elaborate on the fundamentals of best matching in distributed and collaborative systems, by providing general frameworks for (1) Systematic analysis, inclusive taxonomy, analogical and structural comparison between different matching processes; (2) Specification and formulation of problems, and development of algorithms and protocols for best matching; (3) Validation of the models, algorithms, and protocols through extensive numerical experiments and case studies. The first goal is addressed by investigating matching problems in distributed production, manufacturing, supply, and service systems based on a recently developed reference model, the PRISM Taxonomy of Best Matching. Following the second goal, the identified problems are then formulated as mixed-integer programs. Due to the computational complexity of matching problems, various optimization algorithms are developed for solving different problem instances, including modified genetic algorithms, tabu search, and neighbourhood search heuristics. The dynamic and collaborative/competitive behaviors of matching processes in distributed settings are also formulated and examined through various collaboration, best matching, and task administration protocols. In line with the third goal, four case studies are conducted on various manufacturing, supply, and service systems to highlight the impact of best matching on their operational performance, including service level, utilization, stability, and cost-effectiveness, and validate the computational merits of the developed solution methodologies

Algorithms for multi-robot systems on the cooperative exploration & last-mile delivery problems

La aparición de los vehículos aéreos no tripulados (UAVs) y de los vehículos terrestres no tripulados (UGVs) ha llevado a la comunidad científica a enfrentarse a problemas ideando paradigmas de cooperación con UGVs y UAVs. Sin embargo, no suele ser trivial determinar si la cooperación entre UGVs y UAVs es adecuada para un determinado problema. Por esta razón, en esta tesis, investigamos un paradigma particular de cooperación UGV-UAV en dos problemas de la literatura, y proponemos un controlador autónomo para probarlo en escenarios simulados. Primero, formulamos un problema particular de exploración cooperativa que consiste en alcanzar un conjunto de puntos de destino en un área de exploración a gran escala. Este problema define al UGV como una estación de carga móvil para transportar el UAV a través de diferentes lugares desde donde el UAV puede alcanzar los puntos de destino. Por consiguiente, proponemos el algoritmo TERRA para resolverlo. Este algoritmo se destaca por dividir el problema de exploración en cinco subproblemas, en los que cada subproblema se resuelve en una etapa particular del algoritmo. Debido a la explosión de la entrega de paquetes en las empresas de comercio electrónico, formulamos también una generalización del conocido problema de la entrega en la última milla. En este caso, el UGV actúa como una estación de carga móvil que transporta a los paquetes y a los UAVs, y estos se encargan de entregarlos. De esta manera, seguimos la estrategia de división descrita por TERRA, y proponemos el algoritmo COURIER. Este algoritmo replica las cuatro primeras etapas de TERRA, pero construye una nueva quinta etapa para producir un plan de tareas que resuelva el problema. Para evaluar el paradigma de cooperación UGV-UAV en escenarios simulados, proponemos el controlador autónomo ARIES. Este controlador sigue un enfoque jerárquico descentralizado de líder-seguidor para integrar cualquier paradigma de cooperación de manera distribuida. Ambos algoritmos han sido caracterizados para identificar los aspectos relevantes del paradigma de cooperación en los problemas relacionados. Además, ambos demuestran un gran rendimiento del paradigma de cooperación en tales problemas, y al igual que el controlador autónomo, revelan un gran potencial para futuras aplicaciones reales.The emergence of Unmanned Aerial Vehicles (UAVs) and Unmanned Ground Vehicles (UGVs) has conducted the research community to face historical complex problems by devising UGV-UAV cooperation paradigms. However, it is usually not a trivial task to determine whether or not a UGV-UAV cooperation is suitable for a particular problem. For this reason, in this thesis, we investigate a particular UGV-UAV cooperation paradigm over two problems in the literature, and we propose an autonomous controller to test it on simulated scenarios. Driven by the planetary exploration, we formulate a particular cooperative exploration problem consisting of reaching a set of target points in a large-scale exploration area. This problem defines the UGV as a moving charging station to carry the UAV through different locations from where the UAV can reach the target points. Consequently, we propose the cooperaTive ExploRation Routing Algorithm (TERRA) to solve it. This algorithm stands out for splitting up the exploration problem into five sub-problems, in which each sub-problem is solved in a particular stage of the algorithm. In the same way, driven by the explosion of parcels delivery in e-commerce companies, we formulate a generalization of the well-known last-mile delivery problem. This generalization defines the same UGV’s and UAV’s rol as the exploration problem. That is, the UGV acts as a moving charging station which carries the parcels along several UAVs to deliver them. In this way, we follow the split strategy depicted by TERRA to propose the COoperative Unmanned deliveRIEs planning algoRithm (COURIER). This algorithm replicates the first four TERRA’s stages, but it builds a new fifth stage to produce a task plan solving the problem. In order to evaluate the UGV-UAV cooperation paradigm on simulated scenarios, we propose the Autonomous coopeRatIve Execution System (ARIES). This controller follows a hierarchical decentralized leader-follower approach to integrate any cooperation paradigm in a distributed manner. Both algorithms have been characterized to identify the relevant aspects of the cooperation paradigm in the related problems. Also, both of them demonstrate a great performance of the cooperation paradigm in such problems, and as well as the autonomous controller, reveal a great potential for future real applications

Criterios para la planeación de centros de distribución. Revisión bibliométrica

This paper documents the main criteria that have been included for Distribution Centers planning and sizing, through a systematic review; based on the research articles in the SCOPUS bibliographic database. The results obtained show five trends. -The inclusion of automated robots and intelligent forklifts, -the multi-objective distribution centers, -lot sizing, -distribution design or layout, and -planning under uncertainty scenarios. Based on these, important decisions can be made to improve logistics processes, the distribution chain, and the organization’s profitability.Este artículo documenta los principales criterios que se han incluido para la planificación y dimensionamiento de Centros de Distribución, mediante una revisión sistemática; a partir de las publicaciones encontradas en la base de datos bibliográfica SCOPUS. Los resultados obtenidos arrojaron cinco tendencias: -La inclusión de robots automatizados y de montacargas inteligentes, -centros de distribución multiobjetivo, -diseño de la distribución o layout, -el dimensionamiento de lotes y -la planificación bajo escenarios de incertidumbre. Con base en estos, se pueden tomar decisiones importantes para la mejora de los procesos logísticos, la cadena de distribución y la rentabilidad de la organización

Robots learn to behave: improving human-robot collaboration in flexible manufacturing applications

L'abstract è presente nell'allegato / the abstract is in the attachmen

Learning Dynamic Priority Scheduling Policies with Graph Attention Networks

The aim of this thesis is to develop novel graph attention network-based models to automatically learn scheduling policies for effectively solving resource optimization problems, covering both deterministic and stochastic environments. The policy learning methods utilize both imitation learning, when expert demonstrations are accessible at low cost, and reinforcement learning, when otherwise reward engineering is feasible. By parameterizing the learner with graph attention networks, the framework is computationally efficient and results in scalable resource optimization schedulers that adapt to various problem structures. This thesis addresses the problem of multi-robot task allocation (MRTA) under temporospatial constraints. Initially, robots with deterministic and homogeneous task performance are considered with the development of the RoboGNN scheduler. Then, I develop ScheduleNet, a novel heterogeneous graph attention network model, to efficiently reason about coordinating teams of heterogeneous robots. Next, I address problems under the more challenging stochastic setting in two parts. Part 1) Scheduling with stochastic and dynamic task completion times. The MRTA problem is extended by introducing human coworkers with dynamic learning curves and stochastic task execution. HybridNet, a hybrid network structure, has been developed that utilizes a heterogeneous graph-based encoder and a recurrent schedule propagator, to carry out fast schedule generation in multi-round settings. Part 2) Scheduling with stochastic and dynamic task arrival and completion times. With an application in failure-predictive plane maintenance, I develop a heterogeneous graph-based policy optimization (HetGPO) approach to enable learning robust scheduling policies in highly stochastic environments. Through extensive experiments, the proposed framework has been shown to outperform prior state-of-the-art algorithms in different applications. My research contributes several key innovations regarding designing graph-based learning algorithms in operations research.Ph.D