V2V Routing in VANET Based on Heuristic Q-Learning

Designing efficient routing algorithms in vehicular ad hoc networks (VANETs) plays an important role in the emerging intelligent transportation systems. In this paper, a routing algorithm based on the improved Q-learning is proposed for vehicle-to-vehicle (V2V) communications in VANETs. Firstly, a link maintenance time model is established, and the maintenance time is taken as an important parameter in the design of routing algorithm to ensure the reliability of each hop link. Aiming at the low efficiency and slow convergence of Q-learning, heuristic function and evaluation function are introduced to accelerate the update of Q-value of current optimal action, reduce unnecessary exploration, accelerate the convergence speed of Q-learning process and improve learning efficiency. The learning task is dispersed in each vehicle node in the new routing algorithm and it maintains the reliable routing path by periodically exchanging beacon information with surrounding nodes, guides the node’s forwarding action by combining the delay information between nodes to improve the efficiency of data forwarding. The performance of the algorithm is evaluated by NS2 simulator. The results show that the algorithm has a good effect on the package delivery rate and end-to-end delay

Intelligent Reflecting Surface Assisted Anti-Jamming Communications: A Fast Reinforcement Learning Approach

Malicious jamming launched by smart jammers can attack legitimate transmissions, which has been regarded as one of the critical security challenges in wireless communications. With this focus, this paper considers the use of an intelligent reflecting surface (IRS) to enhance anti-jamming communication performance and mitigate jamming interference by adjusting the surface reflecting elements at the IRS. Aiming to enhance the communication performance against a smart jammer, an optimization problem for jointly optimizing power allocation at the base station (BS), and reflecting beamforming at the IRS is formulated while considering quality of service (QoS) requirements of legitimate users. As the jamming model and jamming behavior are dynamic and unknown, a fuzzy win or learn fast-policy hill-climbing (WoLFPHC) learning approach is proposed to jointly optimize the anti-jamming power allocation and reflecting beamforming strategy, where WoLFPHC is capable of quickly achieving the optimal policy without the knowledge of the jamming model, and fuzzy state aggregation can represent the uncertain environment states as aggregate states. Simulation results demonstrate that the proposed anti-jamming learning-based approach can efficiently improve both the IRS-assisted system rate and transmission protection level compared with existing solutions

Impact of an Interfering Node on Unmanned Aerial Vehicle Communications

Unlike terrestrial communications, unmanned aerial vehicle (UAV) communications have some advantages such as the line-of-sight (LoS) environment and flexible mobility. However, the interference will be still inevitable. In this paper, we analyze the effect of an interfering node on the UAV communications by considering the LoS probability and different channel fading for LoS and non-line-of-sight (NLoS) links, which are affected by the elevation angle of the communication link. We then derive a closed-form outage probability in the presence of an interfering node for all the possible scenarios and environments of main and interference links. After discussing the impacts of transmitting and interfering node parameters on the outage probability, we show the existence of the optimal height of the UAV that minimize the outage probability. We also show the NLoS environment can be better than the LoS environment if the average received power of the interference is more dominant than that of the transmitting signal on UAV communications. Finally, we analyze the outage probability for the case of multiple interfering nodes using stochastic geometry and the outage probability of the single interfering node case, and show the effect of the interfering node density on the optimal height of the UAV.Comment: 12 pages, 10 figures, this paper has been submitted in IEEE Transactions on Vehicular Technology. arXiv admin note: substantial text overlap with arXiv:1806.0984