Distributed cloud-edge analytics and machine learning for transportation emissions estimation

(English) In recent years IoT and Smart Cities have become a popular paradigm of computing that is based on network-enabled devices connected providing different functionalities, from sensor measures to domotic actions. With this paradigm, it is possible to provide to the stakeholders near-realtime information of the field, e.g. the current pollution of the city. Along with the mentioned paradigms, Fog Computing enables computation near the sensors where the data is produced, i.e. Edge nodes. This paradigm provides low latency and fault tolerance given the possible independence of the sensor devices. Moreover, pushing this computation enables derived results in a near-realtime fashion. This ability to push the computation to where the data is produced can be beneficial in many situations, however it also requires to include in the Edge the data preparation processes that ensure the fitness for use of the data as the incoming data can be erroneous. Given this situation, Machine Learning can be useful to correct data and also to produce predictions of the future values. Even though there have been studies regarding on the uses of data at the Edge, to our knowledge there is no evaluation of the different modeling situations and the viability of the approach. Therefore, this thesis aims to evaluate the possibility of building a distributed system that ensures the fitness for use of the incoming data through Machine Learning enabled Data Preparation, estimates the emissions and predicts the future status of the city in a near-realtime fashion. We evaluate the viability through three contributions. The first contribution focuses on forecasting in a distributed scenario with road traffic dataset for evaluation. It provides a robust solution to build a central model. This approach is based on Federated Learning, which allows training models at the Edge nodes and then merging them centrally. This way the models in the Edge can be independent but also can be synchronized. The results show the trade-off between accuracy versions training time and a comparison between low-powered devices versus server-class machines. These analyses show that it is viable to use Machine Learning with this paradigm. The second contribution focuses on a particular use case of ship emission estimation. To estimate exhaust emissions data must be correct, which is not always the case. This contribution explores the different techniques available to correct ship registry data and proposes the usage of simple Machine Learning techniques to do imputation of missing or erroneous values. This contribution analyzes the different variables and their relationship to provide the practitioners with guidelines for correction and data treatment. The results show that with classical Machine Learning it is possible to improve the state-of-the-art results. Moreover, as these algorithms are simple enough, they can be used in an Edge device if required. The third contribution focuses on generating new variables from the ones available with a ship trace dataset obtained from the Automatic Identification System (AIS). We use a pipeline of two different methods, a Neural Networks and a clustering algorithm, to group movements into movement patterns or \emph{behaviors}. We test the predicting power of these behaviors to predict ship type, main engine power, and navigational status. The prediction of the main engine power is compared against the standard technique used in ship emission estimation when the ship registry is missing. Our approach was able to detect 45\% of the otherwise undetected emissions if the baseline method was to be used. As ship navigational status is prone to error, the behaviors found are proposed as an alternative variable based in robust data. These contributions build a framework that can distribute the learning processes and that resists network failures in low-powered devices.(Español) En los últimos años, IoT y las Smart Cities se han convertido en un paradigma popular de computación que se basa en dispositivos conectados a la red que proporcionan diferentes funcionalidades, desde medidas de sensores hasta acciones domóticas. Con este paradigma, es posible tener información en casi tiempo real, como por ejemplo la contaminación actual de la ciudad. Junto con los paradigmas mencionados, Fog Computing permite computar cerca de donde se producen los datos, es decir, los nodos Edge. Este paradigma proporciona baja latencia y tolerancia a fallos dada la posible independencia de los dispositivos sensores. Esta posibilidad puede ser beneficiosa en muchas situaciones, sin embargo, requiere incluir en el Edge los procesos de preparación de datos que aseguran la idoneidad para su uso, ya que los datos entrantes pueden ser erróneos. Ante esta situación, el Machine Learning es útil para corregir datos y también para producir predicciones de los valores futuros. A pesar de que se han realizado estudios sobre los usos de los datos en el Edge, hasta donde sabemos, no hay una evaluación de las diferentes situaciones de modelado y la viabilidad del enfoque. Por lo tanto, esta tesis tiene como objetivo evaluar la posibilidad de construir un sistema distribuido que garantice que los datos sean correctos a través de su preparación con Machine Learning. También el sistema deberá estimar las emisiones y predecir el estado futuro de la ciudad de una manera casi en tiempo real. La viabilidad se evalúa a través a través de tres contribuciones. La primera contribución se centra en escenario distribuido con un conjunto de datos de tráfico vial que proporciona una solución robusta para construir un modelo central. Este enfoque se basa en Federated Learning, que permite entrenar modelos en los nodos Edge y luego fusionarlos de forma centralizada. De esta manera, los modelos en el Edge pueden ser independientes, pero también se pueden sincronizar. Los resultados muestran la comparación de la precisión con un modelo central y uno distribuido y una comparación con dispositivos de bajo consumos contra servidores. Estos análisis muestran que es viable utilizar el Machine Learning en este paradigma. La segunda contribución se centra en un caso de uso particular de estimación de las emisiones de barcos. Para estimar las emisiones, los datos deben ser correctos, cosa que no siempre pasa. Esta contribución explora las diferentes técnicas disponibles para corregir los datos del registro de barcos y propone el uso de técnicas simples de Machine Learning para hacer imputación de valores faltantes o erróneos. Esta contribución analiza las diferentes variables y su relación para proporcionar a los profesionales pautas para la corrección y el tratamiento de datos. Los resultados muestran que con el Machine Learning clásico es posible mejorar los resultados frente a métodos del estado del arte. Además, como estos algoritmos son lo suficientemente simples como para poder utilizarse en dispositivos Edge. La tercera contribución se centra en generar nuevas variables a partir de las disponibles con un conjunto de datos de trazabilidad de barcos obtenido del Sistema AIS. Esto se hace utilizando en conjunto una red neuronal y un algoritmo de agrupación para agrupar los movimientos en patrones de movimiento o comportamientos. Se evalúa su funcionamiento para predecir el tipo de barco, la potencia del motor principal y el estado de navegación. Con esta predicción, nuestro sistema es capaz de detectar el 45% de las emisiones que no se detectan con métodos standard. Como el estado de navegación del barco es propenso a errores, los comportamientos encontrados se proponen como una variable alternativa basada en datos robustos. Estas contribuciones constituyen un marco para distribuir los procesos de aprendizaje y que resiste errores en la red con dispositivos de bajo consumo.Arquitectura de computador

Learning user behaviours from website visit profiling

El proyecto consiste en el diseño e implementación de un programa que analiza,a través de los registros o logs, el tráfico y los usuarios de servidores web. En concreto el proyecto pone énfasis en la generación automática de modelos para poder analizar comportamientos de los usuarios

Discovering ship navigation patterns towards environment impact modeling

Ship positioning and maneuvering information is highly relevant to understand the levels of pollution on coastal cities and sea-life quality, containing latent patterns of vessels behavior, that are of utility on earth sciences and environmental research. Using Automatic Identification System (AIS) data enables air quality models to have finer grain estimations. However, the data as it is, carries uncertainty and errors. Therefore, there is a need for a methodology to filter and clean it and to extract patterns. Ship navigation traces can be understood as time series. Here, we present a methodology for characterizing ships by their navigation traces, using Conditional Restricted Boltzmann Machines (CRBMs) plus classic clustering techniques like k-Means. From the inputs received from ships using the AIS, containing ship positions, speed, and characteristics, we produce a processed cruising trace that a CRBM can encode while preserving the time factor and reducing dimensionality of data. Such codification can be then clustered or pattern-mined, then used not only for ship classification but also to cross such behavior patterns with environmental information. In this paper we detail such methodology and validate it using data from the Spanish Ports Authority records from national and international fishing vessels and passenger and cargo ships. Along the pattern mining methodology we propose how to use Apache Spark for the data cleaning process until it arrives to the Conditional Restricted Boltzmann Machine (CRBM). Finally, we develop a visualization tool for data exploration and pattern evaluation

A methodology for the calculation of typical gas concentration values and sampling intervals in the power transformers of a distribution system operator†

Predictive maintenance strategies in power transformers aim to assess the risk through the calculation and monitoring of the health index of the power transformers. The parameter most used in predictive maintenance and to calculate the health index of power transformers is the dissolved gas analysis (DGA). The current tendency is the use of online DGA monitoring equipment while continuing to perform analyses in the laboratory. Although the DGA is well known, there is a lack of published experimental data beyond that in the guides. This study used the nearest-rank method for obtaining the typical gas concentration values and the typical rates of gas increase from a transformer population to establish the optimal sampling interval and alarm thresholds of the continuous monitoring devices for each power transformer. The percentiles calculated by the nearest-rank method were within the ranges of the percentiles obtained using the R software, so this simple method was validated for this study. The results obtained show that the calculated concentration limits are within the range of or very close to those proposed in IEEE C57.104-2019 and IEC 60599:2015. The sampling intervals calculated for each transformer were not correct in all cases since the trend of the historical DGA samples modified the severity of the calculated intervals.This work was partially financed by the EU Regional Development Fund (FEDER) and the Spanish Government under RETOS-COLABORACIÓN RTC-2017-6782-3 and by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 864579 (FLEXIGRID)

Determination of transformer oil contamination from the OLTC gases in the power transformers of a distribution system operator

Power transformers are considered to be the most important assets in power substations. Thus, their maintenance is important to ensure the reliability of the power transmission and distribution system. One of the most commonly used methods for managing the maintenance and establishing the health status of power transformers is dissolved gas analysis (DGA). The presence of acetylene in the DGA results may indicate arcing or high-temperature thermal faults in the transformer. In old transformers with an on-load tap-changer (OLTC), oil or gases can be filtered from the OLTC compartment to the transformer?s main tank. This paper presents a method for determining the transformer oil contamination from the OLTC gases in a group of power transformers for a distribution system operator (DSO) based on the application of the guides and the knowledge of experts. As a result, twenty-six out of the 175 transformers studied are defined as contaminated from the OLTC gases. In addition, this paper presents a methodology based on machine learning techniques that allows the system to determine the transformer oil contamination from the DGA results. The trained model achieves an accuracy of 99.76% in identifying oil contamination.This work was partially financed by the EU Regional Development Fund (FEDER) and the Spanish Government under RETOS-COLABORACIÓN RTC-2017-6782-3 and by the European Union’s Horizon 2020 research and innovation programme under Grant Agreement No. 864579 (FLEXIGRID)

Power quality impact of a small wind energy conversion system connected to the LV grid

This research work is devoted to the study of the Power Quality (PQ) impact of small wind energy conversion systems (SWECS) connected to the low voltage grid. A common coupling point (CCP) has been monitored using a PQ meter that fulfils the standard IEC 61000-4-30 class A. The PQ survey has been conducted with and without the SWECS and the results were compared with the limits defined by the standard EN 50160.This research work was supported by the Spanish Goverment under grant number ENE2007-68032-C04-04, Cantabria Goverment under the R+D 2009 initiative and SONKYO Group

Distributed vs. spot temperature measurements in dynamic rating of overhead power lines

The increase of global energy demand and new ways of electricity production are two of the main challenges for the power sector. The electric market has to address the addition of new and renewable sources of energy to the energy mix and to be able to integrate them into the grid, while maintaining the principles of robustness, security and reliability. All of these changes point to the creation of smart grids, in which advanced generation, information and communication technologies are needed. An accurate knowledge of the electric grid state is crucial for operating the line as efficiently as possible and one of the most important grid parameters to be measured and controlled is the temperature of the overhead conductors due to their relation with the maximum allowable sag of the line and its thermal limit (annealing). This paper presents the results of real-time monitoring of an overhead power line using a distributed temperature sensing system (DTS) and compares these results with spot temperature measurements in order to estimate the loss of accuracy of having less thermal information. This comparison has been carried out in a 30 km long distributed temperature sensing system with fiber optic inside a LA-455 conductor and 6 weather stations placed along the line. An area of influence is defined for each weather station corresponding to the orography of the surroundings. The spot temperatures are obtained from the DTS in the nearest point from the weather stations assuming these six locations to be the ones where the spot temperature measurement equipment would be located. The main conclusion is that, in the case of study, spot measurements are enough to obtain a good approximation of the average temperature of the line conductor

Acoustic noise-based detection of ferroresonance events in isolated neutral power systems with inductive voltage transformers

Power-quality events and operation transients in power systems (PS) with isolated neutral can saturate inductive voltage transformers (IVT), which, when interacting with the overhead and underground cable capacitances, can cause ferroresonance events in the local PS. This abnormal operating mode can partially or totally damage the transformers and switchgears within the affected PS. Distribution system operators (DSO) can minimize these effects by detecting ferroresonance events accurately and fast enough and changing the mode of operation accordingly. Direct detection methods, i.e., based on voltage measurements, are reliable, but the massive deployment of this solution is relatively expensive; i.e., power quality analyzers cost thousands of USD. Alternatively, indirect detection methods are also available, e.g., IVT vibration measurements with accelerometers costing hundreds of USD, but their reliability depends on the installation method used. This manuscript proposes using the acoustic noise caused by magnetostriction forces within the IVT core during ferroresonance events to detect their occurrence. Compared to other indirect methods, electret condenser microphones with preamplifying stage cost less than USD 10 and are less sensitive to the installation procedure. The proposed method is validated experimentally, and its performance compared to IVT vibration measurements one by using the same detection methodology.This work was partially financed by the EU Regional Development Fund (FEDER) and the Spanish Government under RETOS-COLABORACION RTC-2017-6782-3, the Spanish Ministry of Science and Innovation under project PID2021-128941OB-I00, and by the European Union’s Horizon 2020 research and innovation program under grant agreement No. 864579 (FLEXIGRID)

Dynamic rating management of overhead transmission lines operating under multiple weather conditions

Integration of a large number of renewable systems produces line congestions, resulting in a problem for distribution companies, since the lines are not capable of transporting all the energy that is generated. Both environmental and economic constraints do not allow the building new lines to manage the energy from renewable sources, so the efforts have to focus on the existing facilities. Dynamic Rating Management (DRM) of power lines is one of the best options to achieve an increase in the capacity of the lines. The practical application of DRM, based on standards IEEE (Std.738, 2012) and CIGRE TB601 (Technical Brochure 601, 2014) , allows to find several deficiencies related to errors in estimations. These errors encourage the design of a procedure to obtain high accuracy ampacity values. In the case of this paper, two methodologies have been tested to reduce estimation errors. Both methodologies use the variation of the weather inputs. It is demonstrated that a reduction of the conductor temperature calculation error has been achieved and, consequently, a reduction of ampacity error.This research was funded by the Spanish Government AND FEDER funds under the R+D initiative RETOS COLABORACIÓN 2015” with reference RETOS COLABORACIÓN RTC-2015-3795-3 and Spanish R+D initiative with reference ENE2013-42720-R

Thermal behaviour of medium-voltage underground cables under high-load operating conditions

The dynamic management of electric power distribution lines has become a topic of great interest at present. Knowledge of the ampacity of cables is fundamental to carrying out dynamic management. In this study, the ampacity of buried cables in different soil resistivities and depths was calculated. A small-scale model was built in the laboratory to simulate the operating conditions of a buried cable. With the experimental results, a numerical model based on the finite element method was validated to evaluate the ampacities calculated by two standards. A comparison was made between the ampacities calculated from the IEC 60287-1 and UNE 211435 standards and those obtained from the simulated model. In addition, a comparison was made regarding the steady-state temperatures obtained at each calculated ampacity. The results obtained from the simulated model design show that the ampacity calculation method of the IEC 60287-1 standard where drying-out of the soil occurs is the most accurate, and has the least risk of exceeding the maximum permissible cable temperature.This work was financed by the EU Regional Development Fund (FEDER) and the Spanish Government under ENE-2013-42720-R, RETOS-COLABORACION RTC-2015-3795-3 and SODERCAN/FEDER Proyectos Puente 2017 and by the University of Cantabria Industrial Doctorate 19.DI12.649. The authors also acknowledge support received from Viesgo