Multi-Agent Reinforcement Learning for Simulating Pedestrian Navigation

In this paper we introduce a Multi-agent system that uses Reinforcement Learning (RL) techniques to learn local navigational behaviors to simulate virtual pedestrian groups. The aim of the paper is to study empirically the validity of RL to learn agent-based navigation controllers and their transfer capabilities when they are used in simulation environments with a higher number of agents than in the learned scenario. Two RL algorithms which use Vector Quantization (VQ) as the generalization method for the space state are presented. Both strategies are focused on obtaining a good vector quantizier that represents adequately the state space of the agents. We empirically state the convergence of both methods in our navigational Multi-agent learning domain. Besides, we use validation tools of pedestrian models to analyze the simulation results in the context of pedestrian dynamics. The simulations carried out, scaling up the number of agents in our environment (a closed room with a door through which the agents have to leave), have revealed that the basic characteristics of pedestrian movements have been learned

Non-invasive temperature prediction of in vitro therapeutic ultrasound signals using neural networks

In this paper, a novel black-box modelling scheme applied to non-invasive temperature prediction in a homogeneous medium subjected to therapeutic ultrasound is presented. It is assumed that the temperature in a point of the medium is non-linearly related to some spectral features and one temporal feature, extracted from the collected RF-lines. The black-box models used are radial basis functions neural networks (RBFNNs), where the best-fitted models were selected from the space of model structures using a genetic multiobjective strategy. The best-fitted predictive model presents a maximum absolute error less than 0.4 C in a prediction horizon of approximately 2 h, in an unseen data sequence. This work demonstrates that this type of black-box model is well-suited for punctual and noninvasive temperature estimation, achieving, for a single point estimation, better results than the ones presented in the literature, encouraging research on multi-point non-invasive temperature estimation

On the use of artificial neural networks for biomedical applications

Artificial Neural Networks (ANN) are being extensively used in many application areas due to their ability to learn and generalize from data, similarly to a human reaction. This paper reports the use of ANN as a classifier, dynamic model, and diagnosis tool. The examples presented include blood flow emboli classification based on transcranial ultrasound signals, tissue temperature modeling based on imaging transducer’s raw data and identification of ischemic cerebral vascular accident areas based on computer tomography images. In all case studies the performance of ANN proves to produce very accurate results, encouraging the more frequent use of these computational intelligent techniques on medical applications