Comunicación presentada en el Meeting WG3 Database Meeting in Arosa celebrado los días 23-24 de julio en Arosa, Suiza

Hernández Cruz, Bentorey

English

Agencia Estatal de Meteorología

NeuBrewSTSM Boulder, COAuthor: Bentorey Hernández Cruz23/07/2014 www.aemet.es 2NeuBrew: Overview• Overview• Brewer Networks• NeuBrew– David Skaggs Research Center (Boulder, CO)• EuBrewNet– Izaña Atmospheric Research Center (Tenerife, Spain)23/07/2014 www.aemet.es 3NeuBrew: Index• Introduction• System Description• Web Service• Data Transmission• Database• Products Generation• Quality Assurance• Comparison23/07/2014 www.aemet.es 4NeuBrew: Index• Introduction• Network of Brewer Spectrophotometers• Western, Central and Eastern (US)• Measurement• Daily Ultraviolet Radiation• Total Column Ozone• Other equipment• Total Surface Radiation Budget• Total Sky Imagers23/07/2014 www.aemet.es 5NeuBrew: Introduction• Introduction23/07/2014 www.aemet.es 6NeuBrew: System Description • System Description• Centralized server• Quad core• 16 Gigabytes RAM• Hard Drive: 1.8 Terabytes• Os: Centos (Client and Server)• Backups taken every night• Not oriented to real time23/07/2014 www.aemet.es 7NeuBrew: System Description • Web Service• Java + HTML• Apache Tomcat + Servlets23/07/2014 www.aemet.es 8NeuBrew: System Description • Web Service23/07/2014 www.aemet.es 9NeuBrew: System Description • Web Service23/07/2014 www.aemet.es 10NeuBrew: Data Transmission• Data Transmission• Server Request• SSH communications• Sequential mode• One Brewer at a time• Communication problems• Poorly communicated areas23/07/2014 www.aemet.es 11NeuBrew: Database• Database• PostgreSQL (pgAdmin)• Data Issues:• Station• Very descriptive column names• Use of tables for logic behaviour• Configuration of Communications• Different data levels and product generation• Contacts23/07/2014 www.aemet.es 12NeuBrew: Database• DatabaseTABLE stationsFields:"Latitude" and "Longitude" (real): Position of the station"Elevation" (real): Position of the station"Local Timezone" (character varying(30)): Timezone"Phone", "Address”, "Fax", "Mail Code"…TABLE brewerFields:"Notes" (text): Description and information"Active" (boolean): Is the brewer active"In Network" (boolean): Connected to the network23/07/2014 www.aemet.es 13NeuBrew: Database• DatabaseTABLE bfile_scanFields:"Scan Type" (character varying(6)): Type of scan"Data Timestamp" (timestamp(0)): Time of measure"Dark Count" (integer): Dark count"Raw Counts 0" (integer): For every raw countTABLE brewer_file_statusFields:"File Size" (bigint): Size of the received file"Brewer UID" (character varying(6)): Name of the brewer"File Imported into DB" (Boolean): Inserted without problems23/07/2014 www.aemet.es 14NeuBrew: Database• DatabaseTABLE brewer_ingestFields:"Connect Method" (character varying(50)): Internet or dial"Transfer Protocol" (character varying(10)): FTP, XMODEM…"IP Address" (character varying(30)): IP address of the client "Server Port" (character varying(10)): Port for communication (20)VIEW active_brewer_infoFields:"Station Name", Lat", "Long", "Elevation", "Active", "Installed date"Tables:“brewer”, “deployment”, “station”23/07/2014 www.aemet.es 15NeuBrew: Products Generation• Products Generation• UV Index and UV Erythema• From single day to multiple years• Comparison by Brewers and Climate Prediction St.• UV Irradiance• Daily time series• Seven discrete wavelength• UV Spectral Scans• Daily UV Spectral Scan• AM, PM or Both23/07/2014 www.aemet.es 16NeuBrew: Products Generation• Products Generation• Ozone Time Serials• Daily Total-Column Ozone• TOMS/OMI Ozone Level satellite data• Tropospheric Ozone time series• Umkehr Vertical Profile dataset• Ozone Vertical Profiles• Umkehr scans23/07/2014 www.aemet.es 17NeuBrew: Quality Assurance• Quality Assurance• Data processed by the administrator• No automatic way• Some graphics tools showing receptioninformation• Blanks• Medium size of files• Last six days of Ozone and UV Index and Erythema23/07/2014 www.aemet.es 18NeuBrew: Quality Assurance• Quality Assurance• Comprehensive daily report• Experimented administrator• Tables in plain text and messages• ICFs historical from the beginning• Daily backup23/07/2014 www.aemet.es 19NeuBrew: Quality Assurance• Diagnostic• More than twenty five different displays• Langley Regression Analysis tool– Step by step• Brewer UV spike detection and correction– Percentage of spikes in UV spectra• Brewer Multi-instrumental Display– From all brewers in a single page• Ozone QC Level Diagnostic– Differences between data levels23/07/2014 www.aemet.es 20Neubrew: Comparison• ComparisonNeuBrew IberonesiaNetwork not prepared for solving real time issuesNetwork oriented for receiving and processing data in real timeUse of a physical server placed in a laboratory Use of virtual machines and cloud computingDesigned for running in a centralized server Designed for having the possibility to be distributed.Very extended monitoring system for human operatorsReduced monitoring system based in logs and status messagesGraphical information generated in execution time extracting different data levels from databaseGraphic information stored in file systemUse of a reduced number of different technologiesBig amount of different technologies for solving several issuesData transferred by server requests in a scheduled wayClient oriented data transmission depending on local schedulesNeubrewQuestions and Answers23/07/2014 AEMET, Agencia Estatal de Meteorología 21NeubrewThank you for your attention23/07/2014 AEMET, Agencia Estatal de Meteorología 22

NeuBrew

https://repositorio.aemet.es/bitstream/20.500.11765/1139/1/Arosa%202014%20BHC%20Neubrew.pdf

NeuBrew

Abstract

Similar works

Full text

Available Versions

Agencia Estatal de Meteorología