Towards Robust Deep Reinforcement Learning for Traffic Signal Control: Demand Surges, Incidents and Sensor Failures

Reinforcement learning (RL) constitutes a promising solution for alleviating the problem of traffic congestion. In particular, deep RL algorithms have been shown to produce adaptive traffic signal controllers that outperform conventional systems. However, in order to be reliable in highly dynamic urban areas, such controllers need to be robust with the respect to a series of exogenous sources of uncertainty. In this paper, we develop an open-source callback-based framework for promoting the flexible evaluation of different deep RL configurations under a traffic simulation environment. With this framework, we investigate how deep RL-based adaptive traffic controllers perform under different scenarios, namely under demand surges caused by special events, capacity reductions from incidents and sensor failures. We extract several key insights for the development of robust deep RL algorithms for traffic control and propose concrete designs to mitigate the impact of the considered exogenous uncertainties.Comment: 8 page

An efficient genetic algorithm for large-scale planning of robust industrial wireless networks

An industrial indoor environment is harsh for wireless communications compared to an office environment, because the prevalent metal easily causes shadowing effects and affects the availability of an industrial wireless local area network (IWLAN). On the one hand, it is costly, time-consuming, and ineffective to perform trial-and-error manual deployment of wireless nodes. On the other hand, the existing wireless planning tools only focus on office environments such that it is hard to plan IWLANs due to the larger problem size and the deployed IWLANs are vulnerable to prevalent shadowing effects in harsh industrial indoor environments. To fill this gap, this paper proposes an overdimensioning model and a genetic algorithm based over-dimensioning (GAOD) algorithm for deploying large-scale robust IWLANs. As a progress beyond the state-of-the-art wireless planning, two full coverage layers are created. The second coverage layer serves as redundancy in case of shadowing. Meanwhile, the deployment cost is reduced by minimizing the number of access points (APs); the hard constraint of minimal inter-AP spatial paration avoids multiple APs covering the same area to be simultaneously shadowed by the same obstacle. The computation time and occupied memory are dedicatedly considered in the design of GAOD for large-scale optimization. A greedy heuristic based over-dimensioning (GHOD) algorithm and a random OD algorithm are taken as benchmarks. In two vehicle manufacturers with a small and large indoor environment, GAOD outperformed GHOD with up to 20% less APs, while GHOD outputted up to 25% less APs than a random OD algorithm. Furthermore, the effectiveness of this model and GAOD was experimentally validated with a real deployment system

Gumsense - a high power low power sensor node

The development of increasingly complex algorithms for sensor networks has made it difficult for researchers to implement their design on typical sensor network hardware with limited computing resources. The demands on hardware can also mean that small microcontrollers are not the ideal platform for testing computationally and/or memory intensive algorithms. Researchers would also like access to high level programming languages and a wider range of open source libraries. To address this problem we have designed and implemented an architecture, Gumsense which combines a low power micro-controller (8MHz MSP430) with a powerful processor (100-600MHz ARM) on a Gumstix board running Linux. This Open Embedded OS supports a wide variety of programming languages, package management and development tools. A similar hybrid approach was also used in the LEAP platform. The microcontroller wakes up frequently to manage tasks such as activating sensors and gathering data. The intended use-case is to power-up the ARM board and storage only during the brief periods it is needed, for example performing computation or communication

MOSDEN: A Scalable Mobile Collaborative Platform for Opportunistic Sensing Applications

Mobile smartphones along with embedded sensors have become an efficient enabler for various mobile applications including opportunistic sensing. The hi-tech advances in smartphones are opening up a world of possibilities. This paper proposes a mobile collaborative platform called MOSDEN that enables and supports opportunistic sensing at run time. MOSDEN captures and shares sensor data across multiple apps, smartphones and users. MOSDEN supports the emerging trend of separating sensors from application-specific processing, storing and sharing. MOSDEN promotes reuse and re-purposing of sensor data hence reducing the efforts in developing novel opportunistic sensing applications. MOSDEN has been implemented on Android-based smartphones and tablets. Experimental evaluations validate the scalability and energy efficiency of MOSDEN and its suitability towards real world applications. The results of evaluation and lessons learned are presented and discussed in this paper.Comment: Accepted to be published in Transactions on Collaborative Computing, 2014. arXiv admin note: substantial text overlap with arXiv:1310.405

Optimizing the beacon exchange rate for proactive autonomic configuration in ubiquitous MANETs

Proactive self-configuration is indispensable for MANETs like ubiquitous sensor networks (USNs), as component devices of the network are usually exposed to natural or man-made disasters due to the hostile deployment and ad hoc nature of the USNs. Network state beacons (NSBs) are exchanged among the key nodes of the network for crucial and effective monitoring of the network for steady state operation. The rate of beacon exchange (F/sub E/) and its contents, define the time and nature of the proactive action. Therefore it is very important to optimize these parameters to tune the functional response of the USN. This paper presents a comprehensive model for monitoring and proactively reconfiguring the network by optimizing the F/sub E/. The results confirm the improved throughput while maintaining QoS over longer periods of network operation