Improving deep reinforcement learning training convergence using fuzzy logic for autonomous mobile robot navigation

Autonomous robotic navigation has become hotspot research, particularly in complex environments, where inefficient exploration can lead to inefficient navigation. Previous approaches often had a wide range of assumptions and prior knowledge. Adaptations of machine learning (ML) approaches, especially deep learning, play a vital role in the applications of navigation, detection, and prediction about robotic analysis. Further development is needed due to the fast growth of urban megacities. The main problem of training convergence time in deep reinforcement learning (DRL) for mobile robot navigation refers to the amount of time it takes for the agent to learn an optimal policy through trial and error and is caused by the need to collect a large amount of data and computational demands of training deep neural networks. Meanwhile, the assumption of reward in DRL for navigation is problematic as it can be difficult or impossible to define a clear reward function in real-world scenarios, making it challenging to train the agent to navigate effectively. This paper proposes a neuro-symbolic approach that combine the strengths of deep reinforcement learning and fuzzy logic to address the challenges of deep reinforcement learning for mobile robot navigation in terms of training time and the assumption of reward by incorporating symbolic representations to guide the learning process, and inferring the underlying objectives of the task which is expected to reduce the training convergence time

Crowd evacuation with human-level intelligence via neuro-symbolic approach

Understanding human response to crowd emergencies is extremely complex, and it plays a significant role in engineering construction designs and crowd safety. Individual choices, reasoning, and behaviours cannot be fully described by equations or rule-based methods. Accordingly, this research proposes a neuro-symbolic approach for modelling agents with human-level capabilities of reasoning and performance in an emergency evacuation. The proposed neuro-symbolic approach combines deep reinforcement learning with evaluative fuzzy logic to address the challenges of large amounts of required data, time, and trials-and-errors for policy optimization and to handle the assumption of reward function that may not be practical in real scenarios. This neuro-symbolic model has the potential to deal with the complexity of the environment and decision-making process via deep reinforcement learning and enhances the cognitive and visual intelligence via an evaluative fuzzy function, which continuously evaluates agent actions during the training process to boost pedestrian active response to their surroundings, with full awareness of time, thereby, the human-level capacity of reasoning. Moreover, this proposed model optimizes the computational demands of deep reinforcement learning and enables faster learning of new situations. The findings indicate that the proposed model can produce behavioural patterns that align with real observations of crowd evacuation, such as laminar flow, stop-and-go flow, and crowd turbulence. On top of that, a new evacuation behaviour is observed, as some pedestrians avoid congestion at the exit until the density reduces which reflects a level of human reasoning. The proposed model illustrates a higher accuracy and much faster converge than the pure proximal policy optimization model with substantially minimal training timesteps of as little percentage as 2 to 8. Meanwhile, the reliability study records an increase of the mean and standard deviation of evacuation time from 39.7 s, 1.06 to 155.09 s, 7.39 as crowd size increases from 15 to 200 pedestrians, which implies a rise of uncertainty. Therefore, we perceive that this work can provide crowd authorities and construction engineers with insights into complex behaviour and critical conditions to make better evacuation plans and sustainable designs to ensure crowd safety. It also provides a promising alternative to the evident lack of data on critical crowd conditions

Intelligent robot-assisted evacuation: a review

Mass gathering in places such as shopping mall, concerts, sport events etc. is a necessity but may pose threat to the occupants during emergency. This paper presents a review of the different approaches of the crowd evacuation problems which can be classified into classical and modern methods. The classical methods have been widely used which basically depends on prior evacuation warnings or human in assisting an emergent evacuation scenario. Nonetheless, with the lack of important information such as the location of the safest exit, the consequence can be catastrophic. As a result, classical evacuation method becomes tough even though many people might able to assist the evacuees during such situation. Overcoming this, researchers have developed intelligent robots-assisted evacuation systems as a modern approach to manage crowds more systematically and simultaneously during emergency, which will be reviewed in this paper. Finally, this review paper aims to give a broad scope of the reliable evacuation system management to ensure the safety of the evacuees during emergency evacuation

The role of crowd behavior and cooperation strategies during evacuation

Crowd dynamics have constituted a hotspot of research in recent times, particularly in areas where developmental progress has taken place in crowd evacuation for ensuring human safety. In high-density crowd events which happen frequently, panic or an emergency can lead to an increase in congestion which may cause disastrous incidents. Crowd control planning via simulation of people’s movement and behavior can promote safe departures from a space, despite threatening circumstances. Up until now, the evolution of distinctive types of crowd behavior towards cooperative flow remains unexplored. Hence, in this paper, we investigate the impact of potential crowd behavior, namely best-response, risk-seeking, risk-averse, and risk-neutral agents in achieving cooperation during evacuation and its connection with evacuation time using a game-theoretic evacuation simulation model. We analyze the crowd evacuation of a rectangular room with either a single-door or multiple exits in a continuous space. Simulation results show that mutual cooperation during evacuation can be realized when the agents’ population is dominated by risk-averse agents. The results also demonstrate that the risk-seeking agents tend toward aggressiveness by opting for a defector strategy regardless of the local crowd densities, while other crowd behavior shows cooperation under high local crowd densit

A review on crowd analysis of evacuation and abnormality detection based on machine learning systems

Human crowds have become hotspot research, particularly in crowd analysis to ensure human safety. Adaptations of machine learning (ML) approaches, especially deep learning, play a vital role in the applications of evacuation, detection, and prediction pertaining to crowd analysis. Further development in the analysis of crowd is needed to understand human behaviors due to the fast growth of crowd in urban megacities. This article presents a comprehensive review of crowd analysis ML-based systems, where it is categorized with respect to its purposes, viz. crowd evacuation that provides efficient evacuation routes, abnormality detection that could detect the occurrence of any irregular movement or behavior, and crowd prediction that could foresee the occurrence of any possible disasters or predict pedestrian trajectory. Moreover, this article reviews the applied techniques of machine learning with a brief discussion on the used software and simulation platforms. This work also classifies crowd evacuation into data-driven methods and goal-driven learning methods that have attracted significant attention due to their potential to adopt virtual agents with learning capabilities. This review finds that convolutional neural networks and recurrent neural networks have shown superiority in abnormality detection and prediction, whereas deep reinforcement learning has shown potential performance in the development of human level capacities of reasoning. These three methods contribute to the modeling and understanding of pedestrian behavior and will enhance further development in crowd analysis to ensure human safety