Coordinating decentralized learning and conflict resolution across agent boundaries

It is crucial for embedded systems to adapt to the dynamics of open environments. This adaptation process becomes especially challenging in the context of multiagent systems because of scalability, partial information accessibility and complex interaction of agents. It is a challenge for agents to learn good policies, when they need to plan and coordinate in uncertain, dynamic environments, especially when they have large state spaces. It is also critical for agents operating in a multiagent system (MAS) to resolve conflicts among the learned policies of different agents, since such conflicts may have detrimental influence on the overall performance. The focus of this research is to use a reinforcement learning based local optimization algorithm within each agent to learn multiagent policies in a decentralized fashion. These policies will allow each agent to adapt to changes in environmental conditions while reorganizing the underlying multiagent network when needed. The research takes an adaptive approach to resolving conflicts that can arise between locally optimal agent policies. First an algorithm that uses heuristic rules to locally resolve simple conflicts is presented. When the environment is more dynamic and uncertain, a mediator-based mechanism to resolve more complicated conflicts and selectively expand the agents' state space during the learning process is harnessed. For scenarios where mediator-based mechanisms with partially global views are ineffective, a more rigorous approach for global conflict resolution that synthesizes multiagent reinforcement learning (MARL) and distributed constraint optimization (DCOP) is developed. These mechanisms are evaluated in the context of a multiagent tornado tracking application called NetRads. Empirical results show that these mechanisms significantly improve the performance of the tornado tracking network for a variety of weather scenarios. The major contributions of this work are: a state of the art decentralized learning approach that supports agent interactions and reorganizes the underlying network when needed; the use of abstract classes of scenarios/states/actions that efficiently manages the exploration of the search space; novel conflict resolution algorithms of increasing complexity that use heuristic rules, sophisticated automated negotiation mechanisms and distributed constraint optimization methods respectively; and finally, a rigorous study of the interplay between two popular theories used to solve multiagent problems, namely decentralized Markov decision processes and distributed constraint optimization

Risk, Resilience, and Sustainability-Informed Assessment and Management of Aging Structural Systems

During their service life, structural systems (e.g., civil and marine structures) may be subjected to aggressive deteriorations such as corrosion and fatigue and/or extreme events such as floods, collisions, earthquakes, and fires. These deteriorations may start from the day the structures enter in service and, if not effectively managed, can cause a significant reduction in structural functionality and safety. Maintaining performance and functionality of structural systems under these adverse effects is gaining increased attention. This highlights the necessity of effective assessment and management of civil and marine structures in a life-cycle context.The main objective of this study is to develop a risk, sustainability and resilience-informed approach for the life-cycle management of structural systems with emphasis on highway bridges, bridge networks, buildings, interdependent structural systems, and ship structures. Risk - based performance indicators combining the probability of structural failure with the consequences associated with a particular failure event are investigated in this study. Furthermore, a wide range of performance measures is covered under “sustainability” to reflect three aspects: economic, social, and environmental. Sustainability is described as “meeting the needs of present without altering the needs of future generations” (Adams 2006). Sustainability can serve as a useful tool in decision making and risk mitigation associated with civil and marine structures. In addition to risk and sustainability, resilience is another indicator that accounts for structural functionality and recovery patterns after extreme events. Presidential Policy Directive (PPD 2013) defines resilience as “a structure’s ability to prepare for and adapt to changing conditions while simultaneously being able to withstand and recover rapidly from functionality disruptions”. Overall, risk, sustainability, and resilience assessment considering aging and multi-hazard effects are of vital importance to ensure structural safety and functionality of structural systems during their service life.Risk is assessed for highway bridges under the effects of climate change and multiple hazards, including aging effects, flood-induced scour, and earthquake, whereas the adverse effects associated with aging and earthquake are investigated for bridge networks. The sustainability of highway bridges and bridge networks is assessed considering social, economic, and environmental metrics. The seismic resilience of highway bridges under mainshock (MS) only and mainshock-aftershock (MSAS) sequences is investigated to account for structural performance and recovery patterns under extreme events. Additionally, the seismic performance of buildings and interdependent healthcare - bridge network systems is investigated considering correlation effects and uncertainties. Furthermore, a probabilistic methodology to establish optimum pre-earthquake retrofit plans of bridge networks based on risk and sustainability is developed. For ship structures, a decision support system considering structural deteriorations (i.e., corrosion and fatigue) and extreme events (e.g., collision) is established. Specifically, the probabilistic ship collision risk and sustainability are investigated incorporating the attitude of a decision maker. A novel approach is developed to evaluate the time-variant risk of ship structures under corrosion and fatigue during the investigated time interval. Furthermore, a multi-objective optimization problem, which accounts for structural deteriorations and various uncertainties, is formulated to determine optimum inspection planning that reduces the extent of adverse consequence associated with ship failure while simultaneously minimizing the expected total maintenance cost. Additionally, a probabilistic approach for reliability and risk updating of both inspected and uninspected fatigue-sensitive details at both component and system levels is developed considering uncertainties and correlation effects. Overall, this study provides methodologies for the risk, sustainability, and resilience-informed assessment and management of structural systems under structural deteriorations and extreme events in a life-cycle context. Based on the inspection information, the reliability and risk could be updated for the near real-time decision making of deteriorating structures. The proposed probabilistic frameworks are illustrated on highway bridges, bridge networks, buildings, interdependent structural systems, and ship structures. The proposed methodology can be used to assist decision making regarding risk mitigation activities and, ultimately, improve the sustainability of structural systems in a life-cycle context

Integrated platform to assess seismic resilience at the community level

Due to the increasing frequency of disastrous events, the challenge of creating large-scale simulation models has become of major significance. Indeed, several simulation strategies and methodologies have been recently developed to explore the response of communities to natural disasters. Such models can support decision-makers during emergency operations allowing to create a global view of the emergency identifying consequences. An integrated platform that implements a community hybrid model with real-time simulation capabilities is presented in this paper. The platform's goal is to assess seismic resilience and vulnerability of critical infrastructures (e.g., built environment, power grid, socio-technical network) at the urban level, taking into account their interdependencies. Finally, different seismic scenarios have been applied to a large-scale virtual city model. The platform proved to be effective to analyze the emergency and could be used to implement countermeasures that improve community response and overall resilience

Thompson sampling guided stochastic searching on the line for deceptive environments with applications to root-finding problems

publishedVersio

A Router for Symmetrical FPGAs based on Exact Routing Density Evaluation

Abstract This paper presents a new performance and routability driven routing algorithm for symmetrical array based field-programmable gate arrays (FPGAs). A key contribution of our work is to overcome one essential limitation of the previous routing algorithms: inaccurate estimations of routing density which were too general for symmetrical FPGAs. To this end, we derive an exact routing density calculation that is based on a precise analysis of the structure (switch block) of symmetrical FPGAs, and utilize it consistently in global and detailed routings. With an introduction of the proposed accurate routing metrics, we design a new routing algorithm called a cost-effective net-decomposition based routing which is fast, and yet produces remarkable routing results in terms of both routability and path/net delays. We performed an extensive experiment to show the effectiveness of our algorithm based on the proposed cost metrics

遺伝的アルゴリズムの改良に基づくマルチターゲットの運輸問題に関する研究

With the rapid development of economic globalization and information technology, rapid changes have taken place in all fields of society. The status of modern logistics industry in the process of the flow of social means of production and commodities has become increasingly prominent, accompanied by profound changes in production and manufacturing, material circulation, commodity transactions and management methods. Logistics cost accounts for a large share of national GDP, which can reflect the quality and scale of a country\u27s national economy, reduce the logistics cost of enterprises, and greatly improve the profit space. Especially under the background of economic globalization, the competition among enterprises is increasingly fierce, and the impact of logistics on the competitiveness of enterprises is increasingly obvious. In the modern e-commerce environment, with the rapid development of science and technology, the space for enterprises to obtain profits from the products themselves has been greatly reduced. In order to reduce costs and improve profits as much as possible, enterprises focus on logistics. In the whole logistics system, transportation is a very important link. Therefore, efforts to reduce the cost of logistics and transportation can greatly reduce the cost of the entire logistics system. This paper starts from the main factors involved in the transportation logistics, optimizes the main factors affecting the logistics, reduces costs and improves profits.Firstly, this paper discusses and studies the distribution personnel, mainly including the logistics distribution under the limitation of personnel fatigue and the delivery distribution mode under the new mode of personnel allocation - "crowdsourcing logistics". Aiming at the research on the limitation of fatigue, aiming at the maximization of customer satisfaction and the minimization of total cost, this paper constructs a model of path optimization for driver\u27s fatigue driving, and designs a single Partheno-genetic algorithm for the model, which is verified by the distribution case of Japan\u27s otaku. On the research of crowdsourcing delivery, taking the delivery network as the research object, this paper analyzes the distribution process, mode and existing problems of crowdsourcing delivery mode. Based on the purpose of optimizing the distribution network, taking the shortest distribution path and the least time delay as the objective function, the basic optimization model and dynamic optimization model of crowdsourcing distribution path with time window are established, and the rationality of the model is evaluated.Secondly, from the perspective of vehicle research and analysis, mainly study the two-tier node logistics distribution mode based on heterogeneous vehicles. This paper analyzes the common transportation vehicle selection problem in the existing transportation. Based on the genetic algorithm, taking the transportation cost of the double-layer logistics node of a city\u27s seafood products as the optimization goal, and comprehensively considering the problem of taking delivery vehicle route and vehicle configuration strategy of different routes at the same time, the mathematical model of vehicle scheduling and transportation route problem in the double-layer node transportation route is established. In this paper, MATLAB software is used to solve the model based on traditional genetic algorithm and Partheno-genetic algorithm, and the correctness and effectiveness of the model and Partheno-genetic algorithm are verified.Then, from the perspective of transportation path mode, the research mainly involves the current hot "multimodal transport" problem. In this paper, the coal transportation in a country is taken as the research object. Under the mode of "iron water combined transportation", how to reasonably distribute the transportation capacity and correctly select the transportation mode can realize the enterprise to control the logistics cost and ensure the maximum profit. At the same time, based on the traditional genetic algorithm mechanism, aiming at the premature and local search ability of the traditional genetic algorithm in solving the logistics transportation path optimization problem are analyzed Due to the shortage of power, a hybrid genetic algorithm is proposed to solve the model.Finally, the optimization algorithm of logistics distribution is discussed. This paper presents a hybrid genetic algorithm based on information entropy and game theory. First, the initial population is generated by calculating population diversity with information entropy. Combined with parallel genetic algorithm, standard genetic algorithm (SGA), Partheno-genetic algorithm (PGA) and hybrid genetic algorithm (sga-pga) which integrates standard genetic algorithm and Partheno-genetic algorithm (sga-pga) are used to perform evolutionary operations. At the parallel node, information entropy and fitness value of each sub population are used Finally, three programs checking functions Rosenbrock function, Rastrigin function and Schaffer function are introduced to analyze the performance superiority of the algorithm.博士(工学)法政大学 (Hosei University