Soilse: A decentralized approach to optimization of fluctuating urban traffic using Reinforcement Learning

Increasing traffic congestion is a major problem in urban areas, which incurs heavy economic and environmental costs in both developing and developed countries. Efficient urban traffic control (UTC) can help reduce traffic congestion. However, the increasing volume and the dynamic nature of urban traffic pose particular challenges to UTC. Reinforcement Learning (RL) has been shown to be a promising approach to efficient UTC. However, most existing work on RL-based UTC does not adequately address the fluctuating nature of urban traffic. This paper presents Soilse1, a decentralized RL-based UTC optimization scheme that includes a nonparametric pattern change detection mechanism to identify local traffic pattern changes that adversely affect an RL agent's performance. Hence, Soilse is adaptive as agents learn to optimize for different traffic patterns and responsive as agents can detect genuine traffic pattern changes and trigger relearning. We compare the performance of Soilse to two baselines, a fixed-time approach and a saturation balancing algorithm that emulates SCATS, a well-known UTC system. The comparison was performed based on a simulation of traffic in Dublin's inner city centre. Results from using our scheme show an approximate 35%–43% and 40%–54% better performance in terms of average vehicle waiting time and average number of vehicle stops respectively against the best baseline performance in our simulation

IEEE/WIC/ACM International Conference on Intelligent Agent Technology: IAT \u2708

The high growth rate of vehicles per capita now poses a real challenge to efficient Urban Traffic Control (UTC). An efficient solution to UTC must be adaptive in order to deal with the highly-dynamic nature of urban traffic. In the near future, global positioning systems and vehicle-tovehicle/ infrastructure communication may provide a more detailed local view of the traffic situation that could be employed for better global UTC optimization. In this paper we describe the design of a next-generation UTC system that exploits such local knowledge about a junction?s traffic in order to optimize traffic control. Global UTC optimization is achieved using a local Adaptive Round Robin (ARR) phase switching model optimized using Collaborative Reinforcement Learning (CRL). The design employs an ARR-CRL-based agent controller for each signalized junction that collaborates with neighbouring agents in order to learn appropriate phase timing based on the traffic pattern. We compare our approach to non-adaptive fixed-time UTC system and to a saturation balancing algorithm in a largescale simulation of traffic in Dublin?s inner city centre. We show that the ARR-CRL approach can provide significant improvement resulting in up to ~57% lower average waiting time per vehicle compared to the saturation balancing algorithm

UMICS\u2706

Context awareness is a vital element in pervasive and ubiquitous systems. While most existing research has focused on designing context-aware systems to integrate into the environment, less attention has been placed on the interoperability among the entities comprising such systems. In this paper, we consider how the components of a context-aware system can collaborate to achieve a common goal. We provide a taxonomy of such Collaborative Context Awareness (CCA) based on three axis, i.e., goal, approaches and means. We also discuss a number of context-aware systems from different domains, i.e., augmented artefacts, robotics and sensor(/actuator) networks that exhibit some form of collaboration. Finally, we classify the different studied systems according to our taxonomy

Design of a CDD-Based Fault Injection Framework for AUTOSAR Systems

International audienceOver the past years, developing automotive software has been of an Electronic Control Unit (ECU)-specific nature despite the wide range of in-vehicle electronics. With the increasing maintainability cost of such an approach, the AUTomotive Open System Architecture (AUTOSAR) has emerged as a col-lective effort among different elements in the automotive industry in order to provide standardized and open software architecture for different types of vehi-cles. This paper presents a framework design to assess AUTOSAR systems by means of fault injection, which is recommended by the ISO 26262 standard for validating safety requirements at software, system and hardware level. Our pro-posal stems from a number of technical challenges characterizing AUTOSAR systems, and leverages AUTOSAR's Complex Device Driver (CDD) cross-layer and memory partitioning to support the implementation of a minimally intrusive fault injection framework. The potential of the approach in triggering error han-dling mechanisms implemented across the different layers of a given AUTOSAR system is discussed by means of examples

Leveraging Fault Injection Techniques in Critical Industrial Applications

The importance of fault injection techniques is widely recognized by the critical systems industry. Fault injection allows evaluating error handling/mitigation mechanisms and assessing system safety properties under exceptional conditions. Even of more relevance, the use of fault injection is currently recommended by many international standards, such as ISO-26262 and DO-178B, to support the system validation and certification process. This chapter introduces design and technical challenges of fault injection techniques in the context of real industrial applications. Discussion starts from a generic framework that presents the functional components implementing a fault injection campaign. The adoption of the framework to support system evaluation by means of fault injection is shown for Intel Core i7 and AUTOSAR

A collaborative reinforcement learning approach to urban traffic control optimization

The high growth rate of vehicles per capita now poses a real challenge to efficient Urban Traffic Control (UTC). An efficient solution to UTC must be adaptive in order to deal with the highly-dynamic nature of urban traffic. In the near future, global positioning systems and vehicle-tovehicle/ infrastructure communication may provide a more detailed local view of the traffic situation that could be employed for better global UTC optimization. In this paper we describe the design of a next-generation UTC system that exploits such local knowledge about a junction’s traffic in order to optimize traffic control. Global UTC optimization is achieved using a local Adaptive Round Robin (ARR) phase switching model optimized using Collaborative Reinforcement Learning (CRL). The design employs an ARR-CRL-based agent controller for each signalized junction that collaborates with neighbouring agents in order to learn appropriate phase timing based on the traffic pattern. We compare our approach to non-adaptive fixed-time UTC system and to a saturation balancing algorithm in a largescale simulation of traffic in Dublin’s inner city centre. We show that the ARR-CRL approach can provide significant improvement resulting in up to ~57% lower average waiting time per vehicle compared to the saturation balancing algorithm