Autonomous Vehicle Patrolling Through Deep Reinforcement Learning: Learning to Communicate and Cooperate

Autonomous vehicles are suited for continuous area patrolling problems. Finding an optimal patrolling strategy can be challenging due to unknown environmental factors, such as wind or landscape; or autonomous vehicles' constraints, such as limited battery life or hardware failures. Importantly, patrolling large areas often requires multiple agents to collectively coordinate their actions. However, an optimal coordination strategy is often non-trivial to be manually defined due to the complex nature of patrolling environments. In this paper, we consider a patrolling problem with environmental factors, agent limitations, and three typical cooperation problems -- collision avoidance, congestion avoidance, and patrolling target negotiation. We propose a multi-agent reinforcement learning solution based on a reinforced inter-agent learning (RIAL) method. With this approach, agents are trained to develop their own communication protocol to cooperate during patrolling where faults can and do occur. The solution is validated through simulation experiments and is compared with several state-of-the-art patrolling solutions from different perspectives, including the overall patrol performance, the collision avoidance performance, the efficiency of battery recharging strategies, and the overall fault tolerance

An Energy-aware, Fault-tolerant, and Robust Deep Reinforcement Learning based approach for Multi-agent Patrolling Problems

Autonomous vehicles are suited for continuous area patrolling problems. However, finding an optimal patrolling strategy can be challenging for many reasons. Firstly, patrolling environments are often complex and can include unknown environmental factors. Secondly, autonomous vehicles can have failures or hardware constraints, such as limited battery life. Importantly, patrolling large areas often requires multiple agents that need to collectively coordinate their actions. In this work, we consider these limitations and propose an approach based on model-free, deep multi-agent reinforcement learning. In this approach, the agents are trained to automatically recharge themselves when required, to support continuous collective patrolling. A distributed homogeneous multi-agent architecture is proposed, where all patrolling agents execute identical policies locally based on their local observations and shared information. This architecture provides a fault-tolerant and robust patrolling system that can tolerate agent failures and allow supplementary agents to be added to replace failed agents or to increase the overall patrol performance. The solution is validated through simulation experiments from multiple perspectives, including the overall patrol performance, the efficiency of battery recharging strategies, and the overall fault tolerance and robustness

Who Do You Think You\u27re Border Patrolling? : Negotiating Multiracial Identities and Interracial Relationships

Research on racial border patrolling has demonstrated how people police racial borders in order to maintain socially constructed differences and reinforce divisions between racial groups and their members. Existing literature on border patrolling has primarily focused on white/black couples and multiracial families, with discussions contrasting “white border patrolling” and “black border patrolling,” in terms of differential motivations, intentions, and goals (Dalmage 2000). In my dissertation research, I examined a different type of policing racial categories and the spaces in-between these shifting boundaries. I offer up “multiracial interracial border patrolling” as a means of understanding how borderism impacts the lives of “multiracial” individuals in “interracial” relationships. In taking a look at how both identities and relationships involve racial negotiations, I conducted 60 in-depth, face-to-face qualitative interviews with people who indicated having racially mixed parentage or heritage. Respondents shared their experiences of publicly and privately managing their sometimes shifting preferred racial identities; often racially ambiguous appearance; and situationally in/visible “interracial” relationships in an era of colorblind racism. This management included encounters with border patrolling from strangers, significant others, and self. Not only did border patrolling originate from these three sources, but also manifested itself in a variety of forms, including benevolent (positive, supportive); beneficiary (socially and sometimes economically or materially beneficial); protective, and malevolent (negative, malicious, conflictive). Throughout, I discussed the border patrolling variations that “multiracial” individuals in “interracial” relationships face. I also worked to show how people’s participation in border patrolling encouraged their production of colorblind discourses as a strategy for masking their racial attitudes and ideologies about “multiracial” individuals in “interracial” relationships

An Optimal Patrolling Strategy for Tree Networks

We settle a recent conjecture on a continuous patrolling game. In this zero-sum game, an Attacker chooses a time and place to attack a network for a fixed amount of time. A Patroller patrols the network with the aim of intercepting the attack with maximum probability. The conjecture asserts that a particular patrolling strategy called the E-patrolling strategy is optimal for all tree networks. The conjecture was previously known to be true in a limited class of special cases. The E-patrolling strategy has the advantage of being straightforward to calculate and implement. We prove the conjecture by presenting $\varepsilon$-optimal strategies for the Attacker which provide upper bounds for the value of the game that come arbitrarily close to the lower bound provided by the E-patrolling strategy