Reversible Action Design for Combinatorial Optimization with Reinforcement Learning

Combinatorial optimization problem (COP) over graphs is a fundamental challenge in optimization. Reinforcement learning (RL) has recently emerged as a new framework to tackle these problems and has demonstrated promising results. However, most RL solutions employ a greedy manner to construct the solution incrementally, thus inevitably pose unnecessary dependency on action sequences and need a lot of problem-specific designs. We propose a general RL framework that not only exhibits state-of-the-art empirical performance but also generalizes to a variety class of COPs. Specifically, we define state as a solution to a problem instance and action as a perturbation to this solution. We utilize graph neural networks (GNN) to extract latent representations for given problem instances for state-action encoding, and then apply deep Q-learning to obtain a policy that gradually refines the solution by flipping or swapping vertex labels. Experiments are conducted on Maximum $k$-Cut and Traveling Salesman Problem and performance improvement is achieved against a set of learning-based and heuristic baselines

AI/ML Algorithms and Applications in VLSI Design and Technology

An evident challenge ahead for the integrated circuit (IC) industry in the nanometer regime is the investigation and development of methods that can reduce the design complexity ensuing from growing process variations and curtail the turnaround time of chip manufacturing. Conventional methodologies employed for such tasks are largely manual; thus, time-consuming and resource-intensive. In contrast, the unique learning strategies of artificial intelligence (AI) provide numerous exciting automated approaches for handling complex and data-intensive tasks in very-large-scale integration (VLSI) design and testing. Employing AI and machine learning (ML) algorithms in VLSI design and manufacturing reduces the time and effort for understanding and processing the data within and across different abstraction levels via automated learning algorithms. It, in turn, improves the IC yield and reduces the manufacturing turnaround time. This paper thoroughly reviews the AI/ML automated approaches introduced in the past towards VLSI design and manufacturing. Moreover, we discuss the scope of AI/ML applications in the future at various abstraction levels to revolutionize the field of VLSI design, aiming for high-speed, highly intelligent, and efficient implementations

Energy management for electric vehicles in smart cities: a deep learning approach

International audienceWe propose a solution for Electric Vehicle (EV) energy management in smart cities, where a deep learning approach is used to enhance the energy consumption of electric vehicles by trajectory and delay predictions. Two Recurrent Neural Networks are adapted and trained on 60 days of urban traffic. The trained networks show precise prediction of trajec-tory and delay, even for long prediction intervals. An algorithm is designed and applied on well known energy models for traction and air conditioning. We show how it can prevent from a battery exhaustion. Experimental results combining both RNN and energy models demonstrate the efficiency of the proposed solution in terms of route trajectory and delay prediction, enhancing the energy managemen

Urban Air Mobility: Vision, Challenges And Opportunities

Urban Air Mobility (UAM) involving piloted or autonomous aerial vehicles, is envisioned as emerging disruptive technology for next-generation transportation addressing mobility challenges in congested cities. This paradigm may include aircrafts ranging from small unmanned aerial vehicles (UAVs) or drones, to aircrafts with passenger carrying capacity, such as personal air vehicles (PAVs). This paper highlights the UAM vision and brings out the underlying fundamental research challenges and opportunities from computing, networking, and service perspectives for sustainable design and implementation of this promising technology providing an innovative infrastructure for urban mobility. Important research questions include, but are not limited to, real-Time autonomous scheduling, dynamic route planning, aerial-To-ground and inter-vehicle communications, airspace traffic management, on-demand air mobility, resource management, quality of service and quality of experience, sensing (edge) analytics and machine learning for trustworthy decision making, optimization of operational services, and socio-economic impacts of UAM infrastructure on sustainability

Infrastructure systems modeling using data visualization and trend extraction

“Current infrastructure systems modeling literature lacks frameworks that integrate data visualization and trend extraction needed for complex systems decision making and planning. Critical infrastructures such as transportation and energy systems contain interdependencies that cannot be properly characterized without considering data visualization and trend extraction. This dissertation presents two case analyses to showcase the effectiveness and improvements that can be made using these techniques. Case one examines flood management and mitigation of disruption impacts using geospatial characteristics as part of data visualization. Case two incorporates trend analysis and sustainability assessment into energy portfolio transitions. Four distinct contributions are made in this work and divided equally across the two cases. The first contribution identifies trends and flood characteristics that must be included as part of model development. The second contribution uses trend extraction to create a traffic management data visualization system based on the flood influencing factors identified. The third contribution creates a data visualization framework for energy portfolio analysis using a genetic algorithm and fuzzy logic. The fourth contribution develops a sustainability assessment model using trend extraction and time series forecasting of state-level electricity generation in a proposed transition setting. The data visualization and trend extraction tools developed and validated in this research will improve strategic infrastructure planning effectiveness”--Abstract, page iv

Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities

Evolutionary algorithms (EA), a class of stochastic search methods based on the principles of natural evolution, have received widespread acclaim for their exceptional performance in various real-world optimization problems. While researchers worldwide have proposed a wide variety of EAs, certain limitations remain, such as slow convergence speed and poor generalization capabilities. Consequently, numerous scholars actively explore improvements to algorithmic structures, operators, search patterns, etc., to enhance their optimization performance. Reinforcement learning (RL) integrated as a component in the EA framework has demonstrated superior performance in recent years. This paper presents a comprehensive survey on integrating reinforcement learning into the evolutionary algorithm, referred to as reinforcement learning-assisted evolutionary algorithm (RL-EA). We begin with the conceptual outlines of reinforcement learning and the evolutionary algorithm. We then provide a taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the RL-assisted strategy adopted by RL-EA, and its applications according to the existing literature. The RL-assisted procedure is divided according to the implemented functions including solution generation, learnable objective function, algorithm/operator/sub-population selection, parameter adaptation, and other strategies. Finally, we analyze potential directions for future research. This survey serves as a rich resource for researchers interested in RL-EA as it overviews the current state-of-the-art and highlights the associated challenges. By leveraging this survey, readers can swiftly gain insights into RL-EA to develop efficient algorithms, thereby fostering further advancements in this emerging field.Comment: 26 pages, 16 figure