PDRL: Multi-Agent based Reinforcement Learning for Predictive Monitoring

Reinforcement learning has been increasingly applied in monitoring applications because of its ability to learn from previous experiences and can make adaptive decisions. However, existing machine learning-based health monitoring applications are mostly supervised learning algorithms, trained on labels and they cannot make adaptive decisions in an uncertain complex environment. This study proposes a novel and generic system, predictive deep reinforcement learning (PDRL) with multiple RL agents in a time series forecasting environment. The proposed generic framework accommodates virtual Deep Q Network (DQN) agents to monitor predicted future states of a complex environment with a well-defined reward policy so that the agent learns existing knowledge while maximizing their rewards. In the evaluation process of the proposed framework, three DRL agents were deployed to monitor a subject's future heart rate, respiration, and temperature predicted using a BiLSTM model. With each iteration, the three agents were able to learn the associated patterns and their cumulative rewards gradually increased. It outperformed the baseline models for all three monitoring agents. The proposed PDRL framework is able to achieve state-of-the-art performance in the time series forecasting process. The proposed DRL agents and deep learning model in the PDRL framework are customized to implement the transfer learning in other forecasting applications like traffic and weather and monitor their states. The PDRL framework is able to learn the future states of the traffic and weather forecasting and the cumulative rewards are gradually increasing over each episode.Comment: This work has been submitted to the Springer for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessibl

Artificial Intelligence for the Edge Computing Paradigm.

With modern technologies moving towards the internet of things where seemingly every financial, private, commercial and medical transaction being carried out by portable and intelligent devices; Machine Learning has found its way into every smart device and application possible. However, Machine Learning cannot be used on the edge directly due to the limited capabilities of small and battery-powered modules. Therefore, this thesis aims to provide light-weight automated Machine Learning models which are applied on a standard edge device, the Raspberry Pi, where one framework aims to limit parameter tuning while automating feature extraction and a second which can perform Machine Learning classification on the edge traditionally, and can be used additionally for image-based explainable Artificial Intelligence. Also, a commercial Artificial Intelligence software have been ported to work in a client/server setups on the Raspberry Pi board where it was incorporated in all of the Machine Learning frameworks which will be presented in this thesis. This dissertation also introduces multiple algorithms that can convert images into Time-series for classification and explainability but also introduces novel Time-series feature extraction algorithms that are applied to biomedical data while introducing the concept of the Activation Engine, which is a post-processing block that tunes Neural Networks without the need of particular experience in Machine Leaning. Also, a tree-based method for multiclass classification has been introduced which outperforms the One-to-Many approach while being less complex that the One-to-One method.\par The results presented in this thesis exhibit high accuracy when compared with the literature, while remaining efficient in terms of power consumption and the time of inference. Additionally the concepts, methods or algorithms that were introduced are particularly novel technically, where they include: • Feature extraction of professionally annotated, and poorly annotated time-series. • The introduction of the Activation Engine post-processing block. • A model for global image explainability with inference on the edge. • A tree-based algorithm for multiclass classification

Modest : water heater smart control and management

Recent advances in microcontroller technology have given rise to the Internet of Things (IoT), which has created new opportunities in energy automation and optimization. Smart devices and appliances are able to collect operational data, and combine it with data analytics to facilitate optimized control of household electronics. For example, data collected from a smart thermostat can be combined with third-party data sources such as; 15­min interval electricity usage data collected from a smart meter, weather data from numerous sources, utility billing rate data, and data from other smart appliances in the home to optimize the energy usage of residential heating and cooling system. The goals for this project are: 1) Design a control system that enables collection of temperature, flow rate and power data, 2) implement the control system in an electric water heater for residential use, 3) create a connection between a cloud server and the water heater control system for data collection, and 4) develop an advanced control algorithm for water heater thermal management. The main contribution is meant to be a cloud-based infrastructure for IoT aggregation and control, a kind of artificial intelligence for IoT and energy automation.Electrical and Computer Engineerin

A Smart Products Lifecycle Management (sPLM) Framework - Modeling for Conceptualization, Interoperability, and Modularity

Autonomy and intelligence have been built into many of today’s mechatronic products, taking advantage of low-cost sensors and advanced data analytics technologies. Design of product intelligence (enabled by analytics capabilities) is no longer a trivial or additional option for the product development. The objective of this research is aimed at addressing the challenges raised by the new data-driven design paradigm for smart products development, in which the product itself and the smartness require to be carefully co-constructed. A smart product can be seen as specific compositions and configurations of its physical components to form the body, its analytics models to implement the intelligence, evolving along its lifecycle stages. Based on this view, the contribution of this research is to expand the “Product Lifecycle Management (PLM)” concept traditionally for physical products to data-based products. As a result, a Smart Products Lifecycle Management (sPLM) framework is conceptualized based on a high-dimensional Smart Product Hypercube (sPH) representation and decomposition. First, the sPLM addresses the interoperability issues by developing a Smart Component data model to uniformly represent and compose physical component models created by engineers and analytics models created by data scientists. Second, the sPLM implements an NPD3 process model that incorporates formal data analytics process into the new product development (NPD) process model, in order to support the transdisciplinary information flows and team interactions between engineers and data scientists. Third, the sPLM addresses the issues related to product definition, modular design, product configuration, and lifecycle management of analytics models, by adapting the theoretical frameworks and methods for traditional product design and development. An sPLM proof-of-concept platform had been implemented for validation of the concepts and methodologies developed throughout the research work. The sPLM platform provides a shared data repository to manage the product-, process-, and configuration-related knowledge for smart products development. It also provides a collaborative environment to facilitate transdisciplinary collaboration between product engineers and data scientists

Weather Data Transmission Driven By Artificial Neural Network based Prediction

Nowadays, the trend of big data that can be describe as a massive volume and unstructured data have become more complicated because of its difficulty to be process using traditional database and software techniques. Due to increase in size of data, there also demand for big bandwidth for the transmission of big data. Data transmission is important in communication in providing information at different location. In this project we focus on big data transmission in the context of weather data. Weather data is important for meteorologist as it helps them to make weather prediction. Real time weather prediction is really important as it would help in making quick decision to react with the environment and planning for our daily activities. The purpose of this project is to develop a real time and low bandwidth usage for weather data transmission driven by an artificial neural network perform weather forecast using Adaptive Forecasting Model. This project seeks an application context offshore because the data transmission from offshore to onshore is very costly and requires high usage of network bandwidth. Other than that, offshore weather can change rapidly and cause offshore activity to be delayed