A new Data Acquisition and Operation System for Oceanographic ships
and Instrumentation Networks has been developed by the Marine Technology Unit
of the Spanish Research Vessels for their networked remote acquisition platforms.
The new system built over LABVIR project background follows a Service Oriented Architecture
to allow expandable access to acquired data in real time and to archived
data. Several Data layers are implemented to allow data access from the more common
tools and formats used in marine sciences.Peer Reviewe

Afonso, D.

Arilla, E.

Garcia, O.

Hernández, A.

Olivé, J.

Romero, X.

Ruiz, Juan Luis

Río Fernandez, Joaquín del

Sandoval, Antonio

Serrano, J.A.

Sorribes, J.

Trullols Farreny, Enric

English

UPCommons. Portal del coneixement obert de la UPC

Instrumentation Viewpoint 885 m11Abstract - A new Data Acquisition and Operation System for Oceanographic ships and Instrumentation Networks has been developed by the Marine Technology Unit of the Spanish Research Vessels for their networked remote acquisition platforms. The new system built over LABVIR project background follows a Service Oriented Ar-chitecture to allow expandable access to acquired data in real time and to archived data. Several Data layers are implemented to allow data access from the more com-mon tools and formats used in marine sciences. I. INTRODUCTIONThe Marine Technology Unit (UTM) from the Spanish Research Council (CSIC) manages different research platforms as “BO Hesperides”, “BO Sarmiento de Gamboa”, “BO García del Cid” and Antarctic Base “Juan Carlos I”, where different type of data is acquired depending on the instrumentation used in each survey.Broadband communications systems via satellite installed on research plat-forms (ships and bases) provides a link from remote networks to the local area network of the UTM in Barcelona, obtaining a network of instrumentation on each platform that works as a generator node of data,being all of them man-aged from the headquarters of UTM.There is a set of instruments that operate on a continuous and unattended mode, provide information of general interest for surveys, but they provide a valid reference for other specific sensors or information necessary to under-stand and process data from other equipment.The objective is to ensure uniform treatment of information from these systems and their access (in real time or by consulting the historical record) by the maxi-mum possible number of protocols and procedures. A new Oceanographic Data Acquisition and Exploitation System has been integrated on vessels managed by UTM, providing answers submitted to this goal. This system is applicable to any network of instrumentation where an integrated management of data and a wide range of procedures for data access is needed. It has beenIt is designed following a Service Oriented Architecture (SOA) with a multilayer service schema in order to be easily expandable to new required data services.It is fully operational and it has been implemented on top of the Java library and data model build in two technological research projects: The Virtual Distrib-uted Lab for Oceanographic Data Processing and Monitoring (LABVIR) and the Interoperability in Environmental and Marine Sensor Networks (INTIMAS) both funded by the Spanish government [1].II. SYSTEM DESCRIPTIONThe basic mechanisms of transport and exchange of data between acquisition applications and interfaces, users or other external systems have been built us-ing a model of overlapping layers (Figure 1). Basic interfaces, and instrumenta-tion have been placed at the core of the system, and several layers of service of increasing complexity are superimposed until the mechanisms of communica-tion with user applications. Layers of service cover data access in real time as well as access to stored or archived data sets.2.1 Basic Data Transport Service LayerThe basic mechanism for information exchange from acquisition applications to the different layers of service is made using datagrams ASCII following the NMEA-183 standard using the User Datagram Protocol protocol (UDP) in broad-cast mode. The inner ring of the service layer model for communication with in-struments is formed upon this basic mechanism that has been used and tested successfully from several previous works [2].For instruments that use UDP as thea native method of data transmission, this layer of service provides the proper normalization of telegrams that identifies the type of instrument and the platform (ship or base) that is issued. For those that use other input/output interfaces, an additional network interface is pro-vided within this service.The UDP protocol is not as reliable as TCP in the delivery of messages, but is faster, it also introduces low overhead of headers in the network and allows a design between client applications and servers without needing to establish permanent links or specific boot order when is used in broadcast mode.2.2 Access to Historical Records Service LayersIn order to provide access to the historical records of the acquired data a three-layer service have been designed and developed.In each platform and in the central node of the whole system the historic records are feeded by real time acquisition applications while an additional copy is stored, every minute, in the central node by database synchronization procedures. This databases redun-dancy allows the detection of sporadic cuts in satellite links.2.2.1 RAW and Processed Files Service Layer (FILE)This layer provides data storage in ASCII files with the raw information, con-tained in the telegrams NMEA and ASCII files in CSV format [6], with a subset of data encoded within the telegrams specifying the name and units of the mea-sured variables . 2.2.2 The ASCII files are generated daily. The name of each file is a code with information about the day, month and year of its creation..The extension of the name identifies the instrument that originates the data.Relational Database Ser-vice (SQL)This layer provides data storage in tables in a relational database with geo-graphic extensions, as POSTGRES with PostGIS. The acquisition date adjusted to the millisecond is formed as a primary key databases. The relationships between tables are not formally expressed as table relations, but are implemented inside the logic of the service at the application level.2.2.3 Geographic Data Service (WFS and WMS)It is built on the previous layer. This service offers georeferenced data in map and features using the standard WMS and WFS from Open Geospatial Consor-tium [3]. The requests to these two services are initiated by GIS clients using the HTTP protocol as a channel of communication with the servers WMS / WFS. 2.2.4 This is a service typically oriented to recover historical data but its design allows its operation in real time by defining an specific “service” for taking the last data arrived in the system.Data Export Service (EXP)An exportation interface to different formats commonly used in marine and en-vironmental sciences as SensorML [3], NetCDF [4] or openDAP [5] is built from the two previous layers.Real Time Data Access Service LayersThese layers are formed by all the service layers that provides access to data in the same moment of its acquisition, in different formats and network protocols, asynchronously or by request.2.2.5 UDP Data ServiceThis service multiplex UDP messages received on an specific port to multiple ports in broadcast mode or directed to a particular IP address. This service is oriented to processes or applications such as real time data graphic displays or event handlers.2.2.6 TCP Data ServiceThis service replicates the messages received on a specific UDP port to three TCP ports in three different formats: ASCII raw from the received NMEA, self describ-ing XML with information of structure of corresponding data and Java object as a serialized object of data from the used LABVIR library.It is a service oriented to other processes or applications that need to overcome the inherent limitations in the use of UDP, such as the multi attendance to a determined port number in the same server, the reliability of the the data reception, and the overcoming security mechanisms on broadcast traffic related to Java applications.This layer provides an interface to universal queries of on-line data, using a valid TCP client that is a tool from all the operating systems that support TCP/IP as “telnet”.A mULTILAYER SERVICE DATA ACQUISITION AND OPERATION SYS-TEm FOR OCEANOGRAPHIC SHIPS AND INSTRUmENTATION NET-wORKSE. Arilla1, D. Afonso, O. Garcia1, A. Hernández1, J. Olivé1, J. L. Ruiz1, x. Romero1, A. San-doval1 J. A. Serrano1, J. Sorribas1, E. Trullols2, J. del Río2(1)  Departamento de Telematica Unidad de Tecnología marina. CSIC, http://www.utm.csic.es(2)  Dept. Applied mathematics, dept. Electronics, Universitat Politècnica de CatalunyaInstrumentation Viewpoint 8 86m112.2.7 TCP data service implements a mechanism to match the UDP telegram re-ception rate to its asynchronous service to TCP clients.WSS Data ServiceThis service offers real-time data provided by UDP telegrams and the stored data from databases in Service Oriented Access Protocol (SOAP) using a Web Service called Web SADO Server (WSS) that are published on a application server using WSDL [6] for its description.The data access using WSS enables communication from the data acquisition system to any application, regardless of the programming language or platform used.Part of WSS service is built on the SQL service layer becoming a layer from access to historic data layer, and part is built over the Basic Data Transport Service Layer in order to provide data access in Real Time.2.2.8 KMZ Data ServiceThis service provides the values of the last acquired data in compressed KML files [3] whose content and structure is updated in real time and are served through HTTP protocol. In those files  a time value is included in order to allow the automatically refreshment of data.  2.2.9 This is a service oriented to Google Earth clients, although some map serv-ers and Geographical Information System clients can defined contents layers from those files.Data Management Agent Service (NMA)3 This layer allows the asynchronous transmission of data using a TCP channel to a ZABBIX network management [7] server. Through this service, the functional-ity of the management system used to monitor the physical network devices is extended by introducing the quality of data management. Future workIn next future, new instruments will be added to the current integrated data network, specially those who have complex data formats or those whose have a high data streaming behaviour.Data access will be improved by the creation of a new metadata service layer.The system will be implemented in new platforms as cabled seaflooor observa-tories or remote weather stations located in Antarctica.IV. ACKNOWLEDGEMENTSThis work is supported by the Spanish “Ministerio de Ciencia e Innovación” un-der contract CTM2008-04517REFERENCES[1] E. Trullols, J. Sorribas, J. del Rio, C. Samitier, A. Manuel, R. Palomera: LabVir: A Virtual Distrib-uted Measurement System, IEEE Instrumentation and Measurement Technology Conference, Colorado, May 2003.[2] J. Sorribas, A. Tudela, A. Castellón, O. Chic, Z. García, J. Prades, and D. Montero, “An oceano-graphic data acquisition system (ODAS) for Ethernet LAN for Spanish Research vessels,” in Oceans’98 Conference Proceedings (IEEE, ed.), vol. 1, (San Diego (USA)), pp. 93–97, IEEE, 1998[3] Open Geospatial Consortium Standards. http://www.opengeospatial.org/standards/common. Last visit 15/06/09[4] Network Common Data Format. http://www.unidata.ucar.edu/netcdf .Last visited 15/06/09[5] Open-source Project for a Network Data Access Protocol. http://opendap.org/. Last visited 15/06/09[6] Web Service Definition Language (WSDL). http://www.w3.org/TR/wsdl . Last visited 15/06/09[7] An enterprise-class open source distributed monitoring solution. http://www.zabbix.com/. Last visited 15/06/09 Figure 1: Data Service Layers, communication protocols used and data clients used in the Multilayer Service Data Acquisition and Operation System for Oceanographic ships and Instrumentation Net-works. Real time and archive data domains are located at the background.

A multilayer service data acquisition and operation system for oceanographic ships and instrumentation networks

https://upcommons.upc.edu/bitstream/2099/8929/1/A%20multilayer%20service%20data%20acquisition%20and%20operation%20system%20for%20oceanographic%20ships%20and%20instrumentation%20networks.pdf

A multilayer service data acquisition and operation system for oceanographic ships and instrumentation networks

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC