Ant colony system-based applications to electrical distribution system optimization

Chapter 16, February 201

Mobile agent path planning under uncertain environment using reinforcement learning and probabilistic model checking

The major challenge in mobile agent path planning, within an uncertain environment, is effectively determining an optimal control model to discover the target location as quickly as possible and evaluating the control system's reliability. To address this challenge, we introduce a learning-verification integrated mobile agent path planning method to achieve both the effectiveness and the reliability. More specifically, we first propose a modified Q-learning algorithm (a popular reinforcement learning algorithm), called Q EA−learning algorithm, to find the best Q-table in the environment. We then determine the location transition probability matrix, and establish a probability model using the assumption that the agent selects a location with a higher Q-value. Secondly, the learnt behaviour of the mobile agent based on Q EA−learning algorithm, is formalized as a Discrete-time Markov Chain (DTMC) model. Thirdly, the required reliability requirements of the mobile agent control system are specified using Probabilistic Computation Tree Logic (PCTL). In addition, the DTMC model and the specified properties are taken as the input of the Probabilistic Model Checker PRISM for automatic verification. This is preformed to evaluate and verify the control system's reliability. Finally, a case study of a mobile agent walking in a grids map is used to illustrate the proposed learning algorithm. Here we have a special focus on the modelling approach demonstrating how PRISM can be used to analyse and evaluate the reliability of the mobile agent control system learnt via the proposed algorithm. The results show that the path identified using the proposed integrated method yields the largest expected reward.</p

Gaussian Processes for Machine Learning in Robotics

Mención Internacional en el título de doctorNowadays, machine learning is widely used in robotics for a variety of tasks such as perception, control, planning, and decision making. Machine learning involves learning, reasoning, and acting based on the data. This is achieved by constructing computer programs that process the data, extract useful information or features, make predictions to infer unknown properties, and suggest actions to take or decisions to make. This computer program corresponds to a mathematical model of the data that describes the relationship between the variables that represent the observed data and properties of interest. The aforementioned model is learned based on the available training data, which is accomplished using a learning algorithm capable of automatically adjusting the parameters of the model to agree with the data. Therefore, the architecture of the model needs to be selected accordingly, which is not a trivial task and usually depends on the machine-learning engineer’s insights and past experience. The number of parameters to be tuned varies significantly with the selected machine learning model, ranging from two or three parameters for Gaussian processes (GP) to hundreds of thousands for artificial neural networks. However, as more complex and novel robotic applications emerge, data complexity increases and prior experience may be insufficient to define adequate mathematical models. In addition, traditional machine learning methods are prone to problems such as overfitting, which can lead to inaccurate predictions and catastrophic failures in critical applications. These methods provide probabilistic distributions as model outputs, allowing for estimating the uncertainty associated with predictions and making more informed decisions. That is, they provide a mean and variance for the model responses. This thesis focuses on the application of machine learning solutions based on Gaussian processes to various problems in robotics, with the aim of improving current methods and providing a new perspective. Key areas such as trajectory planning for unmanned aerial vehicles (UAVs), motion planning for robotic manipulators and model identification of nonlinear systems are addressed. In the field of path planning for UAVs, algorithms based on Gaussian processes that allow for more efficient planning and energy savings in exploration missions have been developed. These algorithms are compared with traditional analytical approaches, demonstrating their superiority in terms of efficiency when using machine learning. Area coverage and linear coverage algorithms with UAV formations are presented, as well as a sea surface search algorithm. Finally, these algorithms are compared with a new method that uses Gaussian processes to perform probabilistic predictions and optimise trajectory planning, resulting in improved performance and reduced energy consumption. Regarding motion planning for robotic manipulators, an approach based on Gaussian process models that provides a significant reduction in computational times is proposed. A Gaussian process model is used to approximate the configuration space of a robot, which provides valuable information to avoid collisions and improve safety in dynamic environments. This approach is compared to conventional collision checking methods and its effectiveness in terms of computational time and accuracy is demonstrated. In this application, the variance provides information about dangerous zones for the manipulator. In terms of creating models of non-linear systems, Gaussian processes also offer significant advantages. This approach is applied to a soft robotic arm system and UAV energy consumption models, where experimental data is used to train Gaussian process models that capture the relationships between system inputs and outputs. The results show accurate identification of system parameters and the ability to make reliable future predictions. In summary, this thesis presents a variety of applications of Gaussian processes in robotics, from trajectory and motion planning to model identification. These machine learning-based solutions provide probabilistic predictions and improve the ability of robots to perform tasks safely and efficiently. Gaussian processes are positioned as a powerful tool to address current challenges in robotics and open up new possibilities in the field.El aprendizaje automático ha revolucionado el campo de la robótica al ofrecer una amplia gama de aplicaciones en áreas como la percepción, el control, la planificación y la toma de decisiones. Este enfoque implica desarrollar programas informáticos que pueden procesar datos, extraer información valiosa, realizar predicciones y ofrecer recomendaciones o sugerencias de acciones. Estos programas se basan en modelos matemáticos que capturan las relaciones entre las variables que representan los datos observados y las propiedades que se desean analizar. Los modelos se entrenan utilizando algoritmos de optimización que ajustan automáticamente los parámetros para lograr un rendimiento óptimo. Sin embargo, a medida que surgen aplicaciones robóticas más complejas y novedosas, la complejidad de los datos aumenta y la experiencia previa puede resultar insuficiente para definir modelos matemáticos adecuados. Además, los métodos de aprendizaje automático tradicionales son propensos a problemas como el sobreajuste, lo que puede llevar a predicciones inexactas y fallos catastróficos en aplicaciones críticas. Para superar estos desafíos, los métodos probabilísticos de aprendizaje automático, como los procesos gaussianos, han ganado popularidad. Estos métodos ofrecen distribuciones probabilísticas como salidas del modelo, lo que permite estimar la incertidumbre asociada a las predicciones y tomar decisiones más informadas. Esto es, proporcionan una media y una varianza para las respuestas del modelo. Esta tesis se centra en la aplicación de soluciones de aprendizaje automático basadas en procesos gaussianos a diversos problemas en robótica, con el objetivo de mejorar los métodos actuales y proporcionar una nueva perspectiva. Se abordan áreas clave como la planificación de trayectorias para vehículos aéreos no tripulados (UAVs), la planificación de movimientos para manipuladores robóticos y la identificación de modelos de sistemas no lineales. En el campo de la planificación de trayectorias para UAVs, se han desarrollado algoritmos basados en procesos gaussianos que permiten una planificación más eficiente y un ahorro de energía en misiones de exploración. Estos algoritmos se comparan con los enfoques analíticos tradicionales, demostrando su superioridad en términos de eficiencia al utilizar el aprendizaje automático. Se presentan algoritmos de recubrimiento de áreas y recubrimiento lineal con formaciones de UAVs, así como un algoritmo de búsqueda en superficies marinas. Finalmente, estos algoritmos se comparan con un nuevo método que utiliza procesos gaussianos para realizar predicciones probabilísticas y optimizar la planificación de trayectorias, lo que resulta en un rendimiento mejorado y una reducción del consumo de energía. En cuanto a la planificación de movimientos para manipuladores robóticos, se propone un enfoque basado en modelos gaussianos que permite una reducción significativa en los tiempos de cálculo. Se utiliza un modelo de procesos gaussianos para aproximar el espacio de configuraciones de un robot, lo que proporciona información valiosa para evitar colisiones y mejorar la seguridad en entornos dinámicos. Este enfoque se compara con los métodos convencionales de planificación de movimientos y se demuestra su eficacia en términos de tiempo de cálculo y precisión de los movimientos. En esta aplicación, la varianza proporciona información sobre zonas peligrosas para el manipulador. En cuanto a la identificación de modelos de sistemas no lineales, los procesos gaussianos también ofrecen ventajas significativas. Este enfoque se aplica a un sistema de brazo robótico blando y a modelos de consumo energético de UAVs, donde se utilizan datos experimentales para entrenar un modelo de proceso gaussiano que captura las relaciones entre las entradas y las salidas del sistema. Los resultados muestran una identificación precisa de los parámetros del sistema y la capacidad de realizar predicciones futuras confiables. En resumen, esta tesis presenta una variedad de aplicaciones de procesos gaussianos en robótica, desde la planificación de trayectorias y movimientos hasta la identificación de modelos. Estas soluciones basadas en aprendizaje automático ofrecen predicciones probabilísticas y mejoran la capacidad de los robots para realizar tareas de manera segura y eficiente. Los procesos gaussianos se posicionan como una herramienta poderosa para abordar los desafíos actuales en robótica y abrir nuevas posibilidades en el campo.Programa de Doctorado en Ingeniería Eléctrica, Electrónica y Automática por la Universidad Carlos III de MadridPresidente: Juan Jesús Romero Cardalda.- Secretaria: María Dolores Blanco Rojas.- Vocal: Giuseppe Carbon

Models and Issues on Probabilistic Data Streams with Bayesian Networks

This paper proposes the integration of probabilistic data streams and relational database by using Bayesian networks that is one of the most famous techniques for expressing uncertain contexts. A Baysian network is expressed by the graphical model while relational data are expressed by relation. To integrate them we make the relational model as the unified model for its simplicity. A Bayesian network is modeled as an abstract data type in an object relational database, and we define signatures to extract a probabilistic relation from a Bayesian network. We provide a scheme to integrate a probabilistic relation and normal relations. To allow continual queries over streams for a Bayesian network, we introduce a new concept, lifespan.2008 International Symposium on Applications and the Internet : Turku,Finland ; July 28-August 01, 200

Safety Risk Management of LEED Building Construction : A BIM based Approach

Green buildings have been gaining popularity in the construction industry due to their low impact on the environment. Green buildings are aimed at creating energy-efficient, healthy, and environment-friendly buildings. However, OSHA records show that about 48% more accidents occur in green building construction as compared to traditional construction methods. Compromising the workers\u27 health and safety questions the true sustainability of the building. Green buildings have been a popular strategy in institutional sustainability agendas. Globally, LEED is the most popular green buildings rating system. Statistics show that an increasing number of construction projects intend to obtain the LEED certification in the next decade. However, elevated worker health and safety risks have been gradually becoming a concern while pursuing LEED credits. However, there exists a limited study comparing the safety hazards occurring in conventional construction practices and green construction practices.This research explores the major safety risks associated with LEED-certified building construction. Failure Mode Effect, Analysis (FMEA) is used to determine the safety risk associated with each LEED credit. LEED credits were ranked based on safety performance. Safety score and incremental cost of LEED credits were used to identify the optimal credit combination for LEED gold certification that reduces the safety risk and minimizes the cost. Bayesian Belief Networks (BBN) was used to analyze the impact of project factors on safety risk. This analysis identified how the risk level of LEED credits changes based on project parameters. Safety risks identified from FMEA and BBN were used to develop Building Information Modelling (BIM)-based solutions to improve worker safety. The outcomes of this research will address the challenges of LEED construction and inform the construction industry in enhancing the health and safety of construction workers with state-of-the-art technolog