SAT-hadoop-processor: a distributed remote sensing big data processing software for earth observation applications

Nowadays, several environmental applications take advantage of remote sensing techniques. A considerable volume of this remote sensing data occurs in near real-time. Such data are diverse and are provided with high velocity and variety, their pre-processing requires large computing capacities, and a fast execution time is critical. This paper proposes a new distributed software for remote sensing data pre-processing and ingestion using cloud computing technology, specifically OpenStack. The developed software discarded 86% of the unneeded daily files and removed around 20% of the erroneous and inaccurate datasets. The parallel processing optimized the total execution time by 90%. Finally, the software efficiently processed and integrated data into the Hadoop storage system, notably the HDFS, HBase, and Hive.This research was funded by Erasmus+ KA 107 program, and the UPC funded the APC. This work has received funding from the Spanish Government under contracts PID2019-106774RBC21, PCI2019-111851-2 (LeadingEdge CHIST-ERA), PCI2019-111850-2 (DiPET CHIST-ERA).Peer ReviewedPostprint (published version

The 2021 volcanic eruption in La Palma Island and its impact on ionospheric scintillation as measured from GNSS reference stations, GNSS-R, and GNSS-RO

Ionospheric disturbances induced by seismic activity have been studied in the last years by many authors, showing an impact both before and after the occurrence of earthquakes. In this study, the ionospheric scintillation produced by the 2021 La Palma volcano eruption is analyzed. The "Cumbre Vieja" volcano was active from September 19th to December 13th, 2021, and many magnitude 3&ndash;4 earthquakes were recorded, with some of them reaching magnitude 5. In this study the three methods: GNSS reference monitoring, GNSS Reflectometry (GNSS-R) from NASA CYGNSS, and GNSS Radio Occultation (GNSS-RO) from COSMIC and Spire constellations, are used, allowing us to compare and evaluate their performance in the same conditions. To compare the seismic activity with ionospheric scintillation, earthquakes&rsquo; generated energy, and percentile 95 % of the intensity scintillation parameter (S4), measurements have been computed every 6 h intervals for the whole duration of the volcanic eruption. GNSS-RO has shown the best correlation between earthquakes&rsquo; energy and S4, with values up to 0.09 when the perturbations occur around 18 h after the seismic activity. GNSS reference monitoring stations data also shows some correlation 18 h after and 7&ndash;8 days after. As expected, GNSS-R is the one that shows the smallest correlation, as the ionospheric signatures get masked by the signature of the surface where the reflection is taking place. Additionally, as expected as well, the three methods show a smaller correlation during the week before earthquakes.</p

Possible evidence of earthquake precursors observed in ionospheric scintillation events observed from spaceborne GNSS-R data

Several factors may induce perturbations on the ionospheric plasma, changing its average electron density and creating small-scale irregularities, changing its shape and altitude. Solar irradiance and space weather are some of the main factors affecting the ionosphere. They produce a seasonal and daily dependence, modulated by the solar cycle, with more ionospheric activity during periods of higher solar activity. Recent studies shows that another source of perturbations for the ionosphere may be related to internal Earth parameters as seismic activity, in particular, earthquakes. In the period before an earthquake, rocks in the lithosphere are subjected to pressures and movements that may create variations of electromagnetic fields and low frequency waves interacting with the ionosphere. In this work, the ionospheric scintillation intensity index or S4 is estimated from GNSS-R data collected by NASA CYGNSS, and it is correlated with earthquakes events in 2020. Furthermore, it is compared with plasma fluctuation indices measured by ESA Swarm satellites. Two earthquakes in 2020 with magnitudes larger than 7 in the central America region are shown in this work.This work was supported by the Spanish Ministry of Science, Innovation and Universities and EFRD, ”Sensing with Pioneering Opportunistic Techniques” SPOT, grant RTI2018- 099008-BC21/AEI/10.13039/501100011033, and by the Unidad de Excelencia Maria de Maeztu MDM-2016-0600.Peer ReviewedPostprint (author's final draft

First results on the systematic search of land surface temperature anomalies as earthquakes precursors

Every year, earthquakes cause thousands of casualties and high economic losses. For example, in the time frame from 1998 to 2018, the total number of casualties due to earthquakes was larger than 846 thousand people, and the recorded economic losses were about USD 661 billion. At present, there are no earthquake precursors that can be used to trigger a warning. However, some studies have analyzed land surface temperature (LST) anomalies as a potential earthquake precursor. In this study, a large database of global LST data from the Geostationary Operational Environmental Satellite (GOES) and AQUA satellites during the whole year 2020 has been used to study the LST anomalies in the areas affected by earthquakes. A total of 1350 earthquakes with a magnitude larger than M4 were analyzed. Two methods widely used in the literature have been used to detect LST anomalies in the detrended LST time series: the interquartile (IQT) method and the standard deviation (STD). To the authors’ knowledge, it is the first time that the confusion matrix (CM), the receiver operating characteristic curve (ROC), and some other figures of merit (FoM) are used to assess and optimize the performance of the methods, and to select the optimum combination that could be used as a proxy for their occurrence. A positive anomaly was found a few days before the studied earthquakes, followed by the LST decrease after the event. Further studies over larger regions and more extended periods will be needed to consolidate these encouraging results.This work was sponsored by project “GENESIS: GNSS Environmental and Societal Missions—Subproject UPC”, Grant PID2021-126436OB-C21, sponsored by MCIN/AEI/10.13039/501100011033/ and EU ERDF “A way to do Europe”. Badr-Eddine Boudriki Semlali received support in the form of an FI grant: 2021 FI_B 00471 from Generalitat de Catalunya—FI AGAUR 2021.Peer ReviewedPostprint (published version

A preliminary study on ionospheric scintillation anomalies detected using GNSS-R data from NASA CYGNSS mission as possible earthquake precursors

Ionospheric perturbations affect the propagation of electromagnetic waves. These perturbations, besides being a problem for space communications, satellite navigation, and Earth observation techniques, could also be used as another Earth observation tool. Several recent studies showed correlations with earthquakes with ionospheric anomalies, but almost all of them use ground stations to measure the Total Electron Content (TEC) variations, and, in particular, the ones occurring after an earthquake. Here, a preliminary study is presented on how the ionospheric scintillation measured with GNSS-R instruments over oceanic regions shows a small, but detectable correlation with the occurrence of earthquakes, which in some cases occurs before the earthquakes. This study uses GNSS-R data from NASA CYGNSS Mission to measure the ionospheric amplitude scintillation (S4) for 6 months from March 2019 to August 2019, applying a statistical analysis based on confusion matrixes, and the Receiver Operating Characteristic (ROC) curves to correlate S4 anomalous variations to earthquakes. A small positive correlation is found between the ionospheric scintillation and the earthquakes during the six previous days. However, the study has some weakness because (a) a small number (~45) of large (M > 6) earthquakes over oceanic regions are studied, (b) the region studied is close to the geomagnetic equator, where ionospheric scintillations are usual, and (c) the overall correlation is small.This work was supported by the Spanish Ministry of Science, Innovation and Universities and EFRD, “Sensing with Pioneering Opportunistic Techniques” SPOT, grant RTI2018-099008- BC21/AEI/10.13039/501100011033, and by the Unidad de Excelencia Maria de Maeztu MDM-2016- 0600 and, Grant RYC-2016-20918 financed by MCIN/AEI /10.13039/501100011033 and by ESF Investing in your future.Peer ReviewedPostprint (published version

SAT-CEP-monitor: An air quality monitoring software architecture combining complex event processing with satellite remote sensing

La contaminación del aire es un problema importante hoy en día que causa graves daños a la salud humana. Las áreas urbanas son las más afectadas por la degradación de la calidad del aire causada por las emisiones de gases antropogénicos. Aunque existen múltiples propuestas para el monitoreo de la calidad del aire, en la mayoría de los casos, se imponen dos limitaciones: la imposibilidad de procesar datos en tiempo casi real (NRT) para enfoques de teledetección y la imposibilidad de llegar a áreas de acceso limitado o baja cobertura de red para enfoques de datos terrestres. Proponemos una arquitectura de software que combina eficientemente el procesamiento de eventos complejos con datos de teledetección de varios sensores satelitales para monitorear la calidad del aire en NRT, brindando apoyo a los tomadores de decisiones. Ilustramos la solución propuesta calculando los niveles de calidad del aire para varias áreas de Marruecos y España, extrayendo y procesando información satelital en NRT. Este estudio también valida la calidad del aire medida por estaciones terrestres y datos de sensores satelitales.Air pollution is a major problem today that causes serious damage to human health. Urban areas are the most affected by the degradation of air quality caused by anthropogenic gas emissions. Although there are multiple proposals for air quality monitoring, in most cases, two limitations are imposed: the impossibility of processing data in Near Real-Time (NRT) for remote sensing approaches and the impossibility of reaching areas of limited accessibility or low network coverage for ground data approaches. We propose a software architecture that efficiently combines complex event processing with remote sensing data from various satellite sensors to monitor air quality in NRT, giving support to decision-makers. We illustrate the proposed solution by calculating the air quality levels for several areas of Morocco and Spain, extracting and processing satellite information in NRT. This study also validates the air quality measured by ground stations and satellite sensor data.This work was partially supported by the Spanish Ministry of Science and Innovation and the European Regional Development Fund (ERDF) under project FAME [RTI2018-093608-B-C33]. The corresponding author thanks the ERASMUS+ KA107 program for the grant and acknowledges the University of Cadiz for the academic supervision and their research facilities, grant number: 2017-1-ES01- KA107-037422 and 2018-1-ES01-KA107-049705. The authors of this work are also thankful to the Andalusian and Madrid regional governments for providing us with the NRT MGS data

First Results on the Systematic Search of Land Surface Temperature Anomalies as Earthquakes Precursors

Every year, earthquakes cause thousands of casualties and high economic losses. For example, in the time frame from 1998 to 2018, the total number of casualties due to earthquakes was larger than 846 thousand people, and the recorded economic losses were about USD 661 billion. At present, there are no earthquake precursors that can be used to trigger a warning. However, some studies have analyzed land surface temperature (LST) anomalies as a potential earthquake precursor. In this study, a large database of global LST data from the Geostationary Operational Environmental Satellite (GOES) and AQUA satellites during the whole year 2020 has been used to study the LST anomalies in the areas affected by earthquakes. A total of 1350 earthquakes with a magnitude larger than M4 were analyzed. Two methods widely used in the literature have been used to detect LST anomalies in the detrended LST time series: the interquartile (IQT) method and the standard deviation (STD). To the authors&rsquo; knowledge, it is the first time that the confusion matrix (CM), the receiver operating characteristic curve (ROC), and some other figures of merit (FoM) are used to assess and optimize the performance of the methods, and to select the optimum combination that could be used as a proxy for their occurrence. A positive anomaly was found a few days before the studied earthquakes, followed by the LST decrease after the event. Further studies over larger regions and more extended periods will be needed to consolidate these encouraging results

Big data and remote sensing: A new software of ingestion

Currently, remote sensing is widely used in environmental monitoring applications, mostly air quality mapping and climate change supervision. However, satellite sensors occur massive volumes of data in near-real-time, stored in multiple formats and are provided with high velocity and variety. Besides, the processing of satellite big data is challenging. Thus, this study aims to approve that satellite data are big data and proposes a new big data architecture for satellite data processing. The developed software is enabling an efficient remote sensing big data ingestion and preprocessing. As a result, the experiment results show that 86 percent of the unnecessary daily files are discarded with a data cleansing of 20 percent of the erroneous and inaccurate plots. The final output is integrated into the Hadoop system, especially the HDFS, HBase, and Hive, for extra calculation and processing.Peer ReviewedPostprint (published version

Development of a Java-based application for environmental remote sensing data processing

Air pollution is one of the most serious problems the world faces today. It is highly necessary to monitor pollutants in real-time to anticipate and reduce damages caused in several fields of activities. Likewise, it is necessary to provide decision makers with useful and updated environmental data. As a solution to a part of the above-mentioned necessities, we developed a Java-based application software to collect, process and visualize several environmental and pollution data, acquired from the Mediterranean Dialog earth Observatory (MDEO) platform [1]. This application will amass data of Morocco area from EUMETSAT satellites, and will decompress, filter and classify the received datasets. Then we will use the processed data to build an interactive environmental real-time map of Morocco. This should help finding out potential correlations between pollutants and emitting sources

Study of land surface temperature anomalies associated to earthquakes using GOES data

Annually, earthquakes cause human and material losses. For instance, between 1998 and 2018, 846 thousand deaths and about US$ 661 billion of economic losses were recorded due to earthquakes. Currently, there is no clear precursor to forecast earthquakes. However, numerous investigations have attempted to find precursor proxies based on Land Surface Temperature (LST) anomalies. In this study, a big database collected from GOES/ABI instrument during the full year 2020 has been used to calculate the LST anomalies in the earthquakes zones. A total of 1350 earthquakes of Mw = 4 were studied in 2020. Two methods commonly used in the literature, the interquartile method, and the standard deviation of the time series, have been applied to detect LST anomalies. The confusion matrix, some figures of merit, and the receiver operating characteristic curve have been used to evaluate and enhance the performance of the methods and choose the optimum decision threshold. A positive anomaly is usually found before the earthquakes, followed by an LST decrease after the event.This work was supported by project SPOT: Sensing with Pioneering Opportunistic Techniques grant RTI2018- 099008-B-C21/AEI/10.13039/501100011033. Badr-Eddine Boudriki Semlali received support in an FI grant:2021 FI_B 00471 from FI AGAUR 2021.Peer ReviewedPostprint (published version