Development of transportation and supply chain problems with the combination of agent-based simulation and network optimization

Demand drives a different range of supply chain and logistics location decisions, and agent-based modeling (ABM) introduces innovative solutions to address supply chain and logistics problems. This dissertation focuses on an agent-based and network optimization approach to resolve those problems and features three research projects that cover prevalent supply chain management and logistics problems. The first case study evaluates demographic densities in Norway, Finland, and Sweden, and covers how distribution center (DC) locations can be established using a minimizing trip distance approach. Furthermore, traveling time maps are developed for each scenario. In addition, the Nordic area consisting of those three countries is analyzed and five DC location optimization results are presented. The second case study introduces transportation cost modelling in the process of collecting tree logs from several districts and transporting them to the nearest collection point. This research project presents agent-based modelling (ABM) that incorporates comprehensively the key elements of the pick-up and delivery supply chain model and designs the components as autonomous agents communicating with each other. The modelling merges various components such as GIS routing, potential facility locations, random tree log pickup locations, fleet sizing, trip distance, and truck and train transportation. The entire pick-up and delivery operation are modeled by ABM and modeling outcomes are provided by time series charts such as the number of trucks in use, facilities inventory and travel distance. In addition, various scenarios of simulation based on potential facility locations and truck numbers are evaluated and the optimal facility location and fleet size are identified. In the third case study, an agent-based modeling strategy is used to address the problem of vehicle scheduling and fleet optimization. The solution method is employed to data from a real-world organization, and a set of key performance indicators are created to assess the resolution's effectiveness. The ABM method, contrary to other modeling approaches, is a fully customized method that can incorporate extensively various processes and elements. ABM applying the autonomous agent concept can integrate various components that exist in the complex supply chain and create a similar system to assess the supply chain efficiency.Tuotteiden kysyntä ohjaa erilaisia toimitusketju- ja logistiikkasijaintipäätöksiä, ja agenttipohjainen mallinnusmenetelmä (ABM) tuo innovatiivisia ratkaisuja toimitusketjun ja logistiikan ongelmien ratkaisemiseen. Tämä väitöskirja keskittyy agenttipohjaiseen mallinnusmenetelmään ja verkon optimointiin tällaisten ongelmien ratkaisemiseksi, ja sisältää kolme tapaustutkimusta, jotka voidaan luokitella kuuluvan yleisiin toimitusketjun hallinta- ja logistiikkaongelmiin. Ensimmäinen tapaustutkimus esittelee kuinka käyttämällä väestötiheyksiä Norjassa, Suomessa ja Ruotsissa voidaan määrittää strategioita jakelukeskusten (DC) sijaintiin käyttämällä matkan etäisyyden minimoimista. Kullekin skenaariolle kehitetään matka-aikakartat. Lisäksi analysoidaan näistä kolmesta maasta koostuvaa pohjoismaista aluetta ja esitetään viisi mahdollista sijaintia optimointituloksena. Toinen tapaustutkimus esittelee kuljetuskustannusmallintamisen prosessissa, jossa puutavaraa kerätään useilta alueilta ja kuljetetaan lähimpään keräyspisteeseen. Tämä tutkimusprojekti esittelee agenttipohjaista mallinnusta (ABM), joka yhdistää kattavasti noudon ja toimituksen toimitusketjumallin keskeiset elementit ja suunnittelee komponentit keskenään kommunikoiviksi autonomisiksi agenteiksi. Mallinnuksessa yhdistetään erilaisia komponentteja, kuten GIS-reititys, mahdolliset tilojen sijainnit, satunnaiset puunhakupaikat, kaluston mitoitus, matkan pituus sekä monimuotokuljetukset. ABM:n avulla mallinnetaan noutojen ja toimituksien koko ketju ja tuloksena saadaan aikasarjoja kuvaamaan käytössä olevat kuorma-autot, sekä varastomäärät ja ajetut matkat. Lisäksi arvioidaan erilaisia simuloinnin skenaarioita mahdollisten laitosten sijainnista ja kuorma-autojen lukumäärästä sekä tunnistetaan optimaalinen toimipisteen sijainti ja tarvittava autojen määrä. Kolmannessa tapaustutkimuksessa agenttipohjaista mallinnusstrategiaa käytetään ratkaisemaan ajoneuvojen aikataulujen ja kaluston optimoinnin ongelma. Ratkaisumenetelmää käytetään dataan, joka on peräisin todellisesta organisaatiosta, ja ratkaisun tehokkuuden arvioimiseksi luodaan lukuisia keskeisiä suorituskykyindikaattoreita. ABM-menetelmä, toisin kuin monet muut mallintamismenetelmät, on täysin räätälöitävissä oleva menetelmä, joka voi sisältää laajasti erilaisia prosesseja ja elementtejä. Autonomisia agentteja soveltava ABM voi integroida erilaisia komponentteja, jotka ovat olemassa monimutkaisessa toimitusketjussa ja luoda vastaavan järjestelmän toimitusketjun tehokkuuden arvioimiseksi yksityiskohtaisesti.fi=vertaisarvioitu|en=peerReviewed

Mixing quantitative and qualitative methods for sustainable transportation in Smart Cities

L'abstract è presente nell'allegato / the abstract is in the attachmen

Simheuristics to support efficient and sustainable freight transportation in smart city logistics

La logística urbana intel·ligent constitueix un factor crucial en la creació de sistemes de transport urbà eficients i sostenibles. Entre altres factors, aquests sistemes es centren en la incorporació de dades en temps real i en la creació de models de negoci col·laboratius en el transport urbà de mercaderies, considerant l’augment dels habitants en les ciutats, la creixent complexitat de les demandes dels clients i els mercats altament competitius. Això permet als que planifiquen el transport minimitzar els costos monetaris i ambientals del transport de mercaderies a les àrees metropolitanes. Molts problemes de presa de decisions en aquest context es poden formular com a problemes d’optimació combinatòria. Tot i que hi ha diferents enfocaments de resolució exacta per a trobar solucions òptimes a aquests problemes, la seva complexitat i grandària, a més de la necessitat de prendre decisions instantànies pel que fa a l’encaminament de vehicles, la programació o la situació d’instal·lacions, fa que aquestes metodologies no s’apliquin a la pràctica. A causa de la seva capacitat per a trobar solucions pseudoòptimes en gairebé temps real, els algorismes metaheurístics reben una atenció creixent dels investigadors i professionals com a alternatives eficients i fiables per a resoldre nombrosos problemes d’optimació en la creació de la logística de les ciutats intel·ligents. Malgrat el seu èxit, les tècniques metaheurístiques tradicionals no representen plenament la complexitat dels sistemes més realistes. En assumir entrades (inputs) i restriccions de problemes deterministes, la incertesa i el dinamisme experimentats en els escenaris de transport urbà queden sense explicar. Els algorismes simheurístics persegueixen superar aquests inconvenients mitjançant la integració de qualsevol tipus de simulació en processos metaheurístics per a explicar la incertesa inherent a la majoria de les aplicacions de la vida real. Aquesta tesi defineix i investiga l’ús d’algorismes simheurístics com el mètode més adequat per a resoldre problemes d’optimació derivats de la logística de les ciutats. Alguns algorismes simheurístics s’apliquen a una sèrie de problemes complexos, com la recollida de residus urbans, els problemes de disseny de la cadena de subministrament integrada i els models de transport innovadors relacionats amb la col·laboració horitzontal entre els socis de la cadena de subministrament. A més de les discussions metodològiques i la comparació d’algorismes desenvolupats amb els referents de la bibliografia acadèmica, es mostra l’aplicabilitat i l’eficiència dels algorismes simheurístics en diferents casos de gran escala.Las actividades de logística en ciudades inteligentes constituyen un factor crucial en la creación de sistemas de transporte urbano eficientes y sostenibles. Entre otros factores, estos sistemas se centran en la incorporación de datos en tiempo real y la creación de modelos empresariales colaborativos en el transporte urbano de mercancías, al tiempo que consideran el aumento del número de habitantes en las ciudades, la creciente complejidad de las demandas de los clientes y los mercados altamente competitivos. Esto permite minimizar los costes monetarios y ambientales del transporte de mercancías en las áreas metropolitanas. Muchos de los problemas de toma de decisiones en este contexto se pueden formular como problemas de optimización combinatoria. Si bien existen diferentes enfoques de resolución exacta para encontrar soluciones óptimas a tales problemas, su complejidad y tamaño, además de la necesidad de tomar decisiones instantáneas con respecto al enrutamiento, la programación o la ubicación de las instalaciones, hacen que dichas metodologías sean inaplicables en la práctica. Debido a su capacidad para encontrar soluciones pseudoóptimas casi en tiempo real, los algoritmos metaheurísticos reciben cada vez más atención por parte de investigadores y profesionales como alternativas eficientes y fiables para resolver numerosos problemas de optimización en la creación de la logística de ciudades inteligentes. A pesar de su éxito, las técnicas metaheurísticas tradicionales no representan completamente la complejidad de los sistemas más realistas. Al asumir insumos y restricciones de problemas deterministas, se ignora la incertidumbre y el dinamismo experimentados en los escenarios de transporte urbano. Los algoritmos simheurísticos persiguen superar estos inconvenientes integrando cualquier tipo de simulación en procesos metaheurísticos con el fin de considerar la incertidumbre inherente en la mayoría de las aplicaciones de la vida real. Esta tesis define e investiga el uso de algoritmos simheurísticos como método adecuado para resolver problemas de optimización que surgen en la logística de ciudades inteligentes. Se aplican algoritmos simheurísticos a una variedad de problemas complejos, incluyendo la recolección de residuos urbanos, problemas de diseño de la cadena de suministro integrada y modelos de transporte innovadores relacionados con la colaboración horizontal entre los socios de la cadena de suministro. Además de las discusiones metodológicas y la comparación de los algoritmos desarrollados con los de referencia de la bibliografía académica, se muestra la aplicabilidad y la eficiencia de los algoritmos simheurísticos en diferentes estudios de casos a gran escala.Smart city logistics are a crucial factor in the creation of efficient and sustainable urban transportation systems. Among other factors, they focus on incorporating real-time data and creating collaborative business models in urban freight transportation concepts, whilst also considering rising urban population numbers, increasingly complex customer demands, and highly competitive markets. This allows transportation planners to minimize the monetary and environmental costs of freight transportation in metropolitan areas. Many decision-making problems faced in this context can be formulated as combinatorial optimization problems. While different exact solving approaches exist to find optimal solutions to such problems, their complexity and size, in addition to the need for instantaneous decision-making regarding vehicle routing, scheduling, or facility location, make such methodologies inapplicable in practice. Due to their ability to find pseudo-optimal solutions in almost real time, metaheuristic algorithms have received increasing attention from researchers and practitioners as efficient and reliable alternatives in solving numerous optimization problems in the creation of smart city logistics. Despite their success, traditional metaheuristic techniques fail to fully represent the complexity of most realistic systems. By assuming deterministic problem inputs and constraints, the uncertainty and dynamism experienced in urban transportation scenarios are left unaccounted for. Simheuristic frameworks try to overcome these drawbacks by integrating any type of simulation into metaheuristic-driven processes to account for the inherent uncertainty in most real-life applications. This thesis defines and investigates the use of simheuristics as a method of first resort for solving optimization problems arising in smart city logistics concepts. Simheuristic algorithms are applied to a range of complex problem settings including urban waste collection, integrated supply chain design, and innovative transportation models related to horizontal collaboration among supply chain partners. In addition to methodological discussions and the comparison of developed algorithms to state-of-the-art benchmarks found in the academic literature, the applicability and efficiency of simheuristic frameworks in different large-scaled case studies are shown

Modelling human network behaviour using simulation and optimization tools: the need for hybridization

The inclusion of stakeholder behaviour in Operations Research / Industrial Engineering (OR/IE) models has gained much attention in recent years. Behavioural and cognitive traits of people and groups have been integrated in simulation models (mainly through agent-based approaches) as well as in optimization algorithms. However, especially the influence of relations between different actors in human networks is a broad and interdisciplinary topic that has not yet been fully investigated. This paper analyses, from an OR/IE point of view, the existing literature on behaviour-related factors in human networks. This review covers different application fields, including: supply chain management, public policies in emergency situations, and Internet-based human networks. The review reveals that the methodological approach of choice (either simulation or optimization) is highly dependent on the application area. However, an integrated approach combining simulation and optimization is rarely used. Thus, the paper proposes the hybridization of simulation with optimization as one of the best strategies to incorporate human behaviour in human networks and the resulting uncertainty, randomness, and dynamism in related OR/IE models.Peer Reviewe

A DAI approach to modeling the transportation domain

A central problem in the study of autonomous cooperating systems is that of how to establish mechanisms for controlling the interactions between different parts (which are called agents) of the system. One way to integrate such mechanisms into a multi-agent system is to exploit the technique of cooperation or negotiation protocols. In a protocol we distinguish to essential layers: the communication layer specifying the possible flow of messages between different agents, and the decision layer, which controls the selection of a message (speech-act) that the agent sends in a specific situation. In this report we first give a short introduction of our agent model InteRRap which provides the basis for the modeling of the different scenarios considered in the AKA-Mod project at the DFKI. The techniques we will discuss in the following are located in the plan based component and in the cooperation component of this model. The domain of application is the MARS scenario (Modeling a Multi-Agent Scenario for Shipping Companies) which implements a group of shipping companies whose goal it is to deliver a set of dynamically given orders, satisfying a set of given time and/or cost constraints. The complexity of the orders may exceed the capacities of a single company. Therefore, cooperation between companies is required in order to achieve the goal in a satisfactory way. This domain is of considerable interest for studies with economical background as well as for research projects. We give a short summary of results from economical studies that are concerned with the real-world situation in Germany in the transportation domain. They show the need for the development of new techniques from the field of computer science to tackle the problems therein. Then, an overview on related research is presented. Two approaches are discussed in more detail: the first one being based on OR-techniques and a second one being based on the concept of partial intelligent agents attempting to integrate techniques from OR and DAI. Both approaches are concerned with the situation in a single company. However, our purpose to handle the case of distributed shipping companies requires additional mechanisms, e.g. to cope with the problems of task allocation and task decomposition in multi-agent systems. Mechanisms for distributed task decomposition and task allocation processes in multi-agent systems belong to the core of our studies. Therefore, we will first discuss techniques for these problems in a general setting and then describe their implementations in the MARS system. In this description, particular emphasis is placed on the cooperation within a shipping company. Here, one company agent has to allocate a set of orders its truck agents. The truck agents support the company agents by giving cost estimations based on their route planning facility. Thus, this procedure provides the basis for the decisions of the company agents and is discussed in very detail. Finally, we present results from a series of benchmark tests. The test sets have also been run with OR-implementations and thus, give us the opportunity to compare our implementation against these approaches

A new evaluation approach to City Logistics projects A business-oriented Agent-Based model

Supplying goods to urban areas is a fundamental economic process because the majority of the world population lives and buys goods in cities. Freight distribu-tion activities in urban areas account for roughly 40% of supply chain costs and 60% of supply chain CO2 emissions. Moreover, surging e-commerce trends shape the urban freight transportation arena, increasing its complexity and the pressure on private actors. Thus, urban freight transportation activities generate negative externalities, but are relevant to a great amount of enterprises that compose the economic and social fabric. In this context, City Logistics (CL) emerged as a comprehensive concept driv-ing solutions to reduce negative externalities while interfering as little as possible with private actors’ operations and profitability. CL scholars and practitioners are facing several issues arising from e-commerce and population growth. In particu-lar, logistics service providers are called to optimize their operations in order to increase the speed of delivery. At the same time however, CL is dealing with technological and systemic innovation that might enhance optimization capabili-ties and network usage. As a response to the changing environment and within the mandate of CL paradigm, local authorities and private actors have invested on a wide range of initiatives. The variety of approaches adopted and stakeholders involved, at multiple governmental levels, are responsible for a mixed landscape of CL experiences across different regional contexts. Furthermore, despite their relatively large dif-fusion, CL initiatives often fail in taking up after a first pilot implementation, unable to reach paying customers after public subsidies are removed. Therefore, un-derstanding the major business aspects that underline the reasons for adopting CL initiative by private stakeholders is key to a more long-term vision on CL implementation and assessment. Previous research has given little attention to understanding the commercial and business aspects of CL projects before actually designing and implementing them, even though CL scholars have ascertained that evaluation methodologies need to encompass all aspects relevant issues for CL schemes. Several methodol-ogies have been proposed since the inception of CL with the evaluation objective in mind. However, they fall short in different ways. For instance, qualitative methods adopt a short-term feasibility approach to CL evaluation, and the subjec-tive evaluation of quantitative outcomes may potentially influence the ranking between different alternatives. On the other hand, modelling techniques need high quality data to simulate traffic flows and consumers’ demand, but fail short to ad-dress other important decision-making factors related to the business model of stakeholders. Research opportunities therefore lie in mixing the advantages of quantitative and qualitative approaches to include stakeholders in quantitative ex-ante evaluation of CL projects. My thesis will try to answer to the following research questions. -Research question 1: What is the state-of-art of CL projects modelling and evaluation meth-ods/frameworks? -Research question 2: How can an integrated qualitative-quantitative framework for CL evaluation be conceived? -Research question 3: How can a new evaluation framework effectively integrate a business-oriented view of CL systems? The first objective of this thesis is to highlight advantages and disadvantages of assessment methodologies with respect to the integration of the business mo-tives of CL actors into non-project specific, a long-term view on CL project as-sessment. The second objective of this work is to define a theoretical framework for designing and assessing CL projects business models on a qualitative level. To this end, CL systems are here compared to business ecosystems, which are a network of interrelated business entities. In the framework, CL actors can play multi-ple roles, and their decisions are based on their objectives, information, and con-straints. The business model of a business entity within the system is the set of the roles it plays, the business and operative relationships formed with other business entities, and the monetary and intangible values exchanged through these relationships. New quantitative methods are needed for a more sound representation of the patterns emerging from the different behaviours of agents. Hence, the third objective is to build an agent-model proposal for modelling, simulating and ultimately evaluating CL projects business model. In agent-based modelling, each actor can be modelled as an agent possessing objectives and decision-making attributes. Agents act autonomously and their interactions are defined formally by means of ontologies and model narratives built as a representation of real-life system. Finally, an experiment design will be constructed to provide insights on an ex-isting case study related to the introduction of automated parcel locker station. Two CL ecosystem configurations are modelled together in order to simulate the decision to adopt a new logistics service by potential customers. Then, the effect of the decision regarding the allocation of marketing and R&D budget is also evaluated. From the simulation runs, it becomes clear that the outcome for each ecosystem configuration in terms of profits and customers is strongly influenced by the decisions taken within the other configuration. In summary, this thesis provides a first modelling and simulation tool for as-sessing the implications of business model decisions within specific CL business ecosystems. For instance, the strategic decision to adopt a service proposed by a CL company is associated with the evaluation of intangible benefits offered by such company. Moreover, the modelling tool highlights the links between such strategic decisions and the operative ones, such as vehicle routing or inventory policies. Therefore, it proves that qualitative approaches can be used to integrate all stakeholders, while quantitative modelling provide a simulation environment to test long-term effects of different scenarios. However, this study has some limita-tions. For instance, more strategic decisions should be included in the model to investigate endogeneity stemming from agents’ actions. Furthermore, the implica-tion on the business ecosystem of the value of information are not assessed. Finally, the scope of the computational experiment should be widened to include a per-formance evaluation phase, which would then lead to more decision-making by the agents. Further research is aimed at using the tools developed in the thesis to understand how to drive retailers to change their attitude towards CL by under-standing and designing value proposition that might appeal to them. Moreover, the implications of the entrance of new CL players one traditional ones’ business model need to be explored more deeply from the strategic perspective of power relations

Digital twin-driven real-time planning, monitoring, and controlling in food supply chains

There needs to be more clarity about when and how the digital twin approach could benefit the food supply chains. In this study, we develop and solve an integrated problem of procurement, production, and distribution strategies (PPDs) in a medium-scale food processing company. Using the digital twin approach, the model considers the industrial symbiosis opportunities between the supplier, manufacturer, and customer using interval and sequence variables operating in a constrained environment using mixed-integer linear programming (MILP) and agent-based simulation (ABS) methodology. The study optimizes the make-span and lead time, simultaneously achieving a higher level of digitalization. The analysis demonstrates how digital twin accelerates supply chain productivity by improving makespan time, data redundancy (DR), optimal scheduling plan (OSP), overall operations effectiveness (OOE), overall equipment effectiveness (OEE), and capacity utilization. Our findings provide compelling evidence that the seamless integration PPDs enormously enhance production flexibility, resulting in an excellent service level of 94 %. Managers leverage real-time simulation to accurately estimate the replenishment point with minimal lead time, ensuring optimized operations. Furthermore, our results demonstrate that implementing PPDs has yielded considerable benefits. Specifically, we observed a remarkable 65 % utilization of the pasteurizer and aging vessel and an impressive 97 % utilization of the freezer. Moreover, by applying the DT model, the present model found a notable 6 % reduction in backlog, further streamlining operations and enhancing efficiency

Navigation with uncertain spatio-temporal resources

Supporting people with intelligent navigation instructions enables users to efficiently achieve trip-related objectives (e.g., minimum travel time or fuel consumption) and preserves them from making unnecessary detours. This, in turn, enables them to save time, money and, additionally, minimize $CO_2$ emissions. For these reasons, manufacturers integrate navigation systems into almost all modern automobiles. Nevertheless, most of them support only simple routing instructions, i.e., how to drive from location A to B. Albeit, people are regularly faced with more complex decisions, e.g. navigating to a cheap gas station on the route while incorporating dynamic gas price changes. Another example-scenario is after reaching the destination, an available facility to park needs to be found. So far, people cruise almost randomly around the goal area in the search for a parking space. As a consequence, persons valuable time is consumed and unnecessary traffic arises. Besides private persons, transportation companies have to make complex mobility decisions. For instance, taxi drivers have to find out where to move next whenever the taxi is idle. There are plenty possibilities for where the taxi driver could go. In case the last drop-off was in a sparsely populated region, waiting for a call from the taxi office will likely result in a longer drive to the next customer. In turn, customer satisfaction decreases with a longer waiting time and implies a potential loss of customers. Recently, the number of data sources that potentially improve these mobility decisions increased. For instance, on-street parking sensors track the current state of the spaces (e.g. Melbourne), mobile applications collect taxi requests from customers and gas stations publish the current prices all in real-time. This thesis investigates the question of how to design algorithms such that they exploit this volatile data. Standard routing algorithms assume a static world. But the availability of passengers, gas prices and the availability of parking spots change over time in a non-deterministic manner. Hence, we model multiple real-world applications as Markov decision processes (MDP), i.e., a framework for sequential decision making under uncertainty. Depending on the task, we propose to solve the MDP with dynamic programming, replanning and hindsight planning or reinforcement learning. Ultimately, we combine all applications in a single problem domain. Subsequently, we propose a reinforcement learning approach that solves all applications in this domain without modification. Furthermore, it decouples the routing task from solving the application itself. Hence, it is transferable to previously unseen street networks without further training.Durch intelligente Navigationssysteme werden Verkehrsteilnehmer davor bewahrt, Umwege zu fahren. Dadurch sparen sie Zeit, Geld und verringern den $CO_2$-Ausstoß. Aus diesem Grund verbauen Hersteller Navigationssysteme in fast allen Neuwägen. Bis heute unterstützen die meisten Systeme nur einfache Routenplanung, die den kürzesten oder schnellsten Pfad von A nach B berechnen. Dennoch müssen Fahrer regelmäßig Entscheidungen darüber hinaus treffen. Beispielsweise soll eine möglichst günstige Tankstelle auf dem Weg zum eigentlichen Ziel besucht werden. Allerdings kann diese ihre Preise, während der Fahrer oder die Fahrerin auf dem Weg dort hin ist, dynamisch ändern. Anschließend muss, sobald das eigentliche Ziel erreicht ist, ein Parkplatz gefunden werden. Bisher fahren Parkplatzsuchende zufällig durch das Zielgebiet in der Hoffnung möglichst schnell einen freien Parkplatz zu finden. Die Suche verursacht zusätzlichen Verkehr und der Fahrer oder die Fahrerin verbringt mehr Zeit auf der Straße. Neben Privatpersonen müssen auch Transportunternehmen komplexe Entscheidungen über Bewegungen treffen. Zum Beispiel muss ein Taxifahrer, wenn er gerade keinen Fahrgast hat, entscheiden, wo er sich als nächstes positioniert. Zwar könnte er am letzten Zielort warten, bis er einen Anruf der Taxizentrale bekommt. Falls jedoch der letzte Zielort in einem entlegenen Gebiet ist, muss der nächste Fahrgast wahrscheinlich lange warten, bis der Fahrer oder die Fahrerin bei ihm ankommt. Damit sinkt die Kundenzufriedenheit, was wiederum einen potentiellen Verlust der Kunden bedeutet. Seit Kurzem gibt es immer mehr Datenquellen, die Entscheidungen für diese Probleme verbessern. Beispielsweise wird durch Parkplatzsensoren die Verfügbarkeit der Parkplätze verfolgt, mobile Anwendungen sammeln Anfragen über Fahrgäste und Tankstellen veröffentlichen ihren aktuellen Preis in Echtzeit. In dieser Arbeit wird der Forschungsfrage nachgegangen, wie Algorithmen gestaltet werden können, sodass diese veränderlichen Informationen verwendet werden können. Standard-Routing-Algorithmen gehen von einer statischen Welt aus. Aber die Verfügbarkeit von Fahrgästen, die Tankstellenpreise und die Parkplatzzustände ändern sich nicht deterministisch. Aus diesem Grund modellieren wir eine Reihe von Anwendungen als Markov-Entscheidungsproblem (MDP). Applikationsabhängig schlagen wir vor, das MDP mit dynamischer Programmierung, Replanning bzw. Hindsight Planning oder Reinforcement Learning zu lösen. Abschließend fassen wir alle Anwendungen in einer Domäne zusammen. Dadurch können wir einen Reinforcement Learning Ansatz definieren, der alle Anwendungen in dieser Domäne ohne Änderung lösen kann. Dieser Ansatz ermöglicht es, die Routenplanung von der eigentlichen Problemstellung zu lösen. Dadurch ist die gelernte Funktionsapproximation auch auf bisher unbekannte Straßennetze ohne weiteres Training anwendbar