Traffic Signal Control with Cell Transmission Model Using Reinforcement Learning for Total Delay Minimisation

This paper proposes a new framework to control the traffic signal lights by applying the automated goal-directed learning and decision making scheme, namely the reinforcement learning (RL) method, to seek the best possible traffic signal ac- tions upon changes of network state modelled by the signalised cell transmission model (CTM). This paper employs the Q-learning which is one of the RL tools in order to find the traffic signal solution because of its adaptability in finding the real time solu- tion upon the change of states. The goal is for RL to minimise the total network delay. Surprisingly, by using the total network delay as a reward function, the results were not necessarily as good as initially expected. Rather, both simulation and mathemat- ical derivation results confirm that using the newly proposed red light delay as the RL reward function gives better performance than using the total network delay as the reward function. The investigated scenarios include the situations where the summa- tion of overall traffic demands exceeds the maximum flow capacity. Reported results show that our proposed framework using RL and CTM in the macroscopic level can computationally efficiently find the proper control solution close to the brute-forcely searched best periodic signal solution (BPSS). For the practical case study conducted by AIMSUN microscopic traffic simulator, the proposed CTM-based RL reveals that the reduction of the average delay can be significantly decreased by 40% with bus lane and 38% without bus lane in comparison with the case of currently used traffic signal strategy. Therefore, the CTM-based RL algorithm could be a useful tool to adjust the proper traffic signal light in practice

Energy efficient geographical and power based clustering algorithm for heterogeneous wireless sensor networks

Wireless Sensor Network (WSN) has gained tremendous research attention recently due to their extensive applications. As sensor nodes being battery operated, many researchers have made attempts to prolong the lifespan of the WSN by reducing the-per node energy consumption and efficiently utilizing the sensor nodes. However, in the tradition WSNs, nodes were homogeneous and hence could not take full advantage of the presence of heterogeneity in the network. To solve this problem in this paper, we propose Geographical and power based clustering algorithm (GPCA): a heterogeneous-aware clustering protocol, which has significant impact on the entire energy dissipation of WSNs. In GPCA, a Virtual Header (VH) transfers data to the nearest VH, and the nearest VH forwards the data to sink node. In this way, the energy dissipation of the entire network is reduced because of the transmitting distance between VHs and the sink that is greatly shortened. Also, a large number of nodes are self-organized by a distributed cluster formation technique. Moreover, a randomized technique is used to rotate the local cluster-heads base on power label in order to evenly distribute the energy load among the sensors in the network. GPCA uses geographical position to enable scalability and robustness for dynamic networks. By using simulation, the proposed GPCA scheme shows superior performance over the current energy-efficient schemes in terms of network lifespan, Energy dissipation and number of alive nodes

Resource allocation in networks with dynamic topology

EThOS - Electronic Theses Online ServiceGBUnited Kingdo