Calibration of low-cost air pollutant sensors using machine learning techniques

Nowadays concern about air pollution has risen due to the effects of the climate change.The application of machine learning methods for the calibration of low-cost sensors is studied. The short-term, long-term, sensor fusion and training set size needed are analyzed. Thus,considering real scenarios

Comparativa de métodos de localización con Smartphones

Desde la aparición del iPhone en enero del 2007 y de los teléfonos Android en octubre del 2008, el uso de teléfonos inteligentes (smartphones) ha crecido de manera sostenida, de modo que ya forman parte de nuestro día a día. Muchas de las aplicaciones que se desarrollan para smartphones requieren la localización de los usuarios (Location-Based Applications). Las librerías de soporte al desarrollo (Software Developement Kit, SDK) de Android y de iOS proporcionan métodos de localización (por ejemplo: LocationManager en Android o CLLocationManager en iOS). Sin embargo, en muchas ocasiones las soluciones ofrecidas por estos SDKs no son óptimas, bien por una falta de precisión en la localización, o por un excesivo consumo de batería. Por ello, en este proyecto se investiga cómo se puede combinar la información proporcionada por los servicios estándares de los SDKs con la información de otros sensores (ej. acelerómetro, giroscopio), para así obtener técnicas de localización que pueden ser más útiles y eficientes en muchos casos y compararlas con las comúnmente usadas en la actualidad.Des de l’aparició de l’iPhone el gener del 2007 i dels telèfons Android l’octubre del 2008, l’ús de telèfons intel·ligents (smartphones) ha crescut de manera sostinguda, de manera que ja formen part del nostre dia a dia. Moltes aplicacions que es desenvolupen per a smartphones requereixen de la localització dels usuaris (Location-Based Application). Les llibreries de suport al desenvolupament (Software Developement Kit, SDK) de Android i de iOS proporcionen mètodes de localització (per exemple: LocationManager en Android o CLLocationManager en iOS). Però, en moltes ocasions les solucions oferides per aquests SDKs no són òptimes, ja sigui per una falta de precisió en la localització, o per un excessiu consum de bateria. Per això, en aquest projecte s’investiga com es pot combinar la informació proporcionada pels serveis estàndards dels SDK amb la informació d’altres sensors (ex: acceleròmetre, giroscopi), per així obtenir tècniques de localització que poden ser més útils i eficients en molts casos i comparar-les amb les normalment utilitzades en l’actualitat.Since the release of the iPhone in January 2007 and the release of the Android phones in October 2008, the use of the smartphones has grown steadily, so that they are already part of our daily life. Lots of applications developed for smartphones require the users localization (Location-Based Application). The Android and iOS software developement kits (SDK) offer localization methods (for example: LocationManager for Android and CLLocationManager for iOS). But in most cases the solution offered by the SDKs is not optimal, either because they lack of precision or they have an excessive power consumption. For that, this project investigates how we can combine the information given by the standard services of the SDKs with other sensor information (for example: accelerometers and gyroscopes), in order to obtain localization techniques that can be more useful and more efficient in a lot of cases and compare them with the commonly used nowadays

A comparative study of calibration methods for low-cost ozone sensors in IoT platforms

© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.This paper shows the result of the calibration process of an Internet of Things platform for the measurement of tropospheric ozone (O 3 ). This platform, formed by 60 nodes, deployed in Italy, Spain, and Austria, consisted of 140 metal–oxide O 3 sensors, 25 electro-chemical O 3 sensors, 25 electro-chemical NO 2 sensors, and 60 temperature and relative humidity sensors. As ozone is a seasonal pollutant, which appears in summer in Europe, the biggest challenge is to calibrate the sensors in a short period of time. In this paper, we compare four calibration methods in the presence of a large dataset for model training and we also study the impact of a limited training dataset on the long-range predictions. We show that the difficulty in calibrating these sensor technologies in a real deployment is mainly due to the bias produced by the different environmental conditions found in the prediction with respect to those found in the data training phase.Peer ReviewedPostprint (author's final draft

Graph signal reconstruction techniques for IoT air pollution monitoring platforms

Air pollution monitoring platforms play a very important role in preventing and mitigating the effects of pollution. Recent advances in the field of graph signal processing have made it possible to describe and analyze air pollution monitoring networks using graphs. One of the main applications is the reconstruction of the measured signal in a graph using a subset of sensors. Reconstructing the signal using information from neighboring sensors is a key technique for maintaining network data quality, with examples including filling in missing data with correlated neighboring nodes, creating virtual sensors, or correcting a drifting sensor with neighboring sensors that are more accurate. This paper proposes a signal reconstruction framework for air pollution monitoring data where a graph signal reconstruction model is superimposed on a graph learned from the data. Different graph signal reconstruction methods are compared on actual air pollution data sets measuring O3, NO2, and PM10. The ability of the methods to reconstruct the signal of a pollutant is shown, as well as the computational cost of this reconstruction. The results indicate the superiority of methods based on kernel-based graph signal reconstruction, as well as the difficulties of the methods to scale in an air pollution monitoring network with a large number of low-cost sensors. However, we show that the scalability of the framework can be improved with simple methods, such as partitioning the network using a clustering algorithm.This work is supported by the National Spanish funding PID2019-107910RB-I00, by regional project 2017SGR-990, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu.Peer ReviewedPostprint (author's final draft

Machine Learning (ML) module

Lectures notes of the machine learning content of the course TOML (Topics on Optimization and Machine Learning) at Master in Innovation and Research in Informatics (MIRI) at FIB, UPC.2023/202

Review of linear algebra and applications to data science

Lectures notes of the "Review of linear algebra and applications to data science" of the course SANS (Statistical Analysis of Networks and Systems) at Master in Innovation and Research in Informatics (MIRI) at FIB, UPC.2023/202

Volterra graph-based outlier detection for air pollution sensor networks

Today's air pollution sensor networks pose new challenges given their heterogeneity of low-cost sensors and high-cost instrumentation. Recently, with the advent of graph signal processing, sensor network measurements have been successfully represented by graphs depicting the relationships between sensors. However, one of the main problems of these sensor networks is their reliability, especially due to the inclusion of low-cost sensors, so the detection and identification of outliers is extremely important for maintaining the quality of the network data. In order to better identify the outliers of the sensors composing a network, we propose the Volterra graph-based outlier detection (VGOD) mechanism, which uses a graph learned from data and a Volterra-like graph signal reconstruction model to detect and localize abnormal measurements in air pollution sensor networks. The proposed unsupervised decision process is compared with other outlier detection methods, state-of-the-art graph-based methods and non-graph-based methods, showing improvements in both detection and localization of anomalous measurements, so that anomalous measurements can be corrected and malfunctioning sensors can be replaced.This work is supported by the National Spanish funding PID2019-107910RB-I00, by regional project 2017SGR-990, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu.Peer ReviewedPostprint (author's final draft

Review of probability theory

Lectures notes of the review of probability theory of the course SANS (Statistical Analysis of Networks and Systems) at Master in Innovation and Research in Informatics (MIRI) at FIB, UPC.2023/202

Raw data collected from NO2, O3 and NO air pollution electrochemical low-cost sensors

Recently, the monitoring of air pollution by means of lowcost sensors has become a growing research field due to the study of techniques based on machine learning to improve the sensors’ data quality. For this purpose, sensors undergo a calibration process, where these are placed in-situ nearby a regulatory reference station. The data set explained in this paper contains data from two self-built low-cost air pollution nodes deployed for four months, from January 16, 2021 to May 15, 2021, at an official air quality reference station in Barcelona, Spain. The goal of the deployment was to have five electrochemical sensors at a high sampling rate of 0.5 Hz; two NO2 sensors, two O3 sensors, and one NO sensor. It should be noted that the reference stations publish air pollution data every hour, thus at a rate of 2.7 × 10-4 Hz. In addition, the nodes have also captured temperature and relative humidity data, which are typically used as correctors in the calibration of low-cost sensors. The availability of the sensors’ time series at this high resolution is important in order to be able to carry out analysis from the signal processing perspective, allowing the study of sensor sampling strategies, sensor signal filtering, and the calibration of low-cost sensors among others.This work was supported by National Spanish project PID2019-107910RB-I00, and regional project 2017SGR-990, and with the support of Secretaria d’Universitats i Recerca de la Generalitat de Catalunya i del Fons Social Europeu.Peer ReviewedPostprint (published version

Temporal pattern-based denoising and calibration for low-cost sensors in IoT monitoring platforms

The introduction of low-cost sensors (LCSs) in air quality Internet of Things (IoT) monitoring platforms presents the challenge of improving the quality of the data that these sensors provide. In this article, we propose two algorithms to perform denoising and calibration for LCSs used in IoT monitoring platforms. Sensors are first calibrated in situ using linear or nonlinear machine learning models that only take into account instantaneous measurements. The best calibration model is used to estimate the values measured by the sensor during the sensor deployment. To improve the values of the estimates produced by the in situ calibration model, we propose to take into account the temporal patterns present in signals, such as temperature or tropospheric ozone that have regular patterns, e.g., daily. The first method, which we call temporal pattern-based denoising (TPB-D), performs signal denoising by projecting the daily signals of the in situ calibrated LCS onto a subspace generated by the daily signals stored in a database taken by reference instruments. The second method, which we call temporal pattern-based calibration (TPB-C), considers that if we also have a reference instrument colocated to the LCSs over a period of time, we can correct with a linear mapping with regularization the daily LCS signals projected in the subspace produced by the reference database to be as similar as possible to the projected signals of the colocated reference instrument. The results show that the TPB-D improves the estimates made by in situ calibration by up to 10%–20%, while the TPB-C improves the estimates made by in situ calibration by up to 20%–40%.This work was supported in part by the National Spanish Funding under Grant PID2019-107910RB-I00 and in part by the Regional Project under Grant 2021 SGR 01059.Peer ReviewedPostprint (author's final draft