Automatic processing, quality assurance and serving of real-time weather data

Recent advances in technology have produced a significant increase in the availability of free sensor data over the Internet. With affordable weather monitoring stations now available to individual meteorology enthusiasts a reservoir of real time data such as temperature, rainfall and wind speed can now be obtained for most of the United States and Europe. Despite the abundance of available data, obtaining useable information about the weather in your local neighbourhood requires complex processing that poses several challenges. This paper discusses a collection of technologies and applications that harvest, refine and process this data, culminating in information that has been tailored toward the user. In this case we are particularly interested in allowing a user to make direct queries about the weather at any location, even when this is not directly instrumented, using interpolation methods. We also consider how the uncertainty that the interpolation introduces can then be communicated to the user of the system, using UncertML, a developing standard for uncertainty representation

The Design and Operation of The Keck Observatory Archive

The Infrared Processing and Analysis Center (IPAC) and the W. M. Keck Observatory (WMKO) operate an archive for the Keck Observatory. At the end of 2013, KOA completed the ingestion of data from all eight active observatory instruments. KOA will continue to ingest all newly obtained observations, at an anticipated volume of 4 TB per year. The data are transmitted electronically from WMKO to IPAC for storage and curation. Access to data is governed by a data use policy, and approximately two-thirds of the data in the archive are public.Comment: 12 pages, 4 figs, 4 tables. Presented at Software and Cyberinfrastructure for Astronomy III, SPIE Astronomical Telescopes + Instrumentation 2014. June 2014, Montreal, Canad

Automated reduction of submillimetre single-dish heterodyne data from the James Clerk Maxwell Telescope using ORAC-DR

With the advent of modern multi-detector heterodyne instruments that can result in observations generating thousands of spectra per minute it is no longer feasible to reduce these data as individual spectra. We describe the automated data reduction procedure used to generate baselined data cubes from heterodyne data obtained at the James Clerk Maxwell Telescope. The system can automatically detect baseline regions in spectra and automatically determine regridding parameters, all without input from a user. Additionally it can detect and remove spectra suffering from transient interference effects or anomalous baselines. The pipeline is written as a set of recipes using the ORAC-DR pipeline environment with the algorithmic code using Starlink software packages and infrastructure. The algorithms presented here can be applied to other heterodyne array instruments and have been applied to data from historical JCMT heterodyne instrumentation.Comment: 18 pages, 13 figures, submitted to Monthly Notices of the Royal Astronomical Societ

Paediatric radiology seen from Africa. Part I: providing diagnostic imaging to a young population

Article approval pendingPaediatric radiology requires dedicated equipment, specific precautions related to ionising radiation, and specialist knowledge. Developing countries face difficulties in providing adequate imaging services for children. In many African countries, children represent an increasing proportion of the population, and additional challenges follow from extreme living conditions, poverty, lack of parental care, and exposure to tuberculosis, HIV, pneumonia, diarrhoea and violent trauma. Imaging plays a critical role in the treatment of these children, but is expensive and difficult to provide. The World Health Organisation initiatives, of which the World Health Imaging System for Radiography (WHIS-RAD) unit is one result, needs to expand into other areas such as the provision of maintenance servicing. New initiatives by groups such as Rotary and the World Health Imaging Alliance to install WHIS-RAD units in developing countries and provide digital solutions, need support. Paediatric radiologists are needed to offer their services for reporting, consultation and quality assurance for free by way of teleradiology. Societies for paediatric radiology are needed to focus on providing a volunteer teleradiology reporting group, information on child safety for basic imaging, guidelines for investigations specific to the disease spectrum, and solutions for optimising imaging in children

Big Data and the Internet of Things

Advances in sensing and computing capabilities are making it possible to embed increasing computing power in small devices. This has enabled the sensing devices not just to passively capture data at very high resolution but also to take sophisticated actions in response. Combined with advances in communication, this is resulting in an ecosystem of highly interconnected devices referred to as the Internet of Things - IoT. In conjunction, the advances in machine learning have allowed building models on this ever increasing amounts of data. Consequently, devices all the way from heavy assets such as aircraft engines to wearables such as health monitors can all now not only generate massive amounts of data but can draw back on aggregate analytics to "improve" their performance over time. Big data analytics has been identified as a key enabler for the IoT. In this chapter, we discuss various avenues of the IoT where big data analytics either is already making a significant impact or is on the cusp of doing so. We also discuss social implications and areas of concern.Comment: 33 pages. draft of upcoming book chapter in Japkowicz and Stefanowski (eds.) Big Data Analysis: New algorithms for a new society, Springer Series on Studies in Big Data, to appea

Environmental conditions of a salt-marsh biodiversity experiment on the island of Spiekeroog (Germany)

Field experiments investigating biodiversity and ecosystem functioning require the observation of abiotic parameters, especially when carried out in the intertidal zone. An experiment for biodiversity–ecosystem functioning was set up in the intertidal zone of the back-barrier salt marsh of Spiekeroog Island in the German Bight. Here, we report the accompanying instrumentation, maintenance, data acquisition, data handling and data quality control as well as monitoring results observed over a continuous period from September 2014 to April 2017. Time series of abiotic conditions were measured at several sites in the vicinity of newly built experimental salt-marsh islands on the tidal flat. Meteorological measurements were conducted from a weather station (WS, https://doi.org/10.1594/PANGAEA.870988), oceanographic conditions were sampled through a bottom-mounted recording current meter (RCM, https://doi.org/10.1594/PANGAEA.877265) and a bottom-mounted tide and wave recorder (TWR, https://doi.org/10.1594/PANGAEA.877258). Tide data are essential in calculating flooding duration and flooding frequency with respect to different salt-marsh elevation zones. Data loggers (DL) for measuring the water level (DL-W, https://doi.org/10.1594/PANGAEA.877267), temperature (DL-T, https://doi.org/10.1594/PANGAEA.877257), light intensity (DL-L, https://doi.org/10.1594/PANGAEA.877256) and conductivity (DL-C, https://doi.org/10.1594/PANGAEA.877266) were deployed at different elevational zones on the experimental islands and the investigated salt-marsh plots. A data availability of 80% for 17 out of 23 sensors was achieved. Results showed the influence of seasonal and tidal dynamics on the experimental islands. Nearby salt-marsh plots exhibited some differences, e.g., in temperature dynamics. Thus, a consistent, multi-parameter, long-term dataset is available as a basis for further biodiversity and ecosystem functioning studies

The space physics environment data analysis system (SPEDAS)

With the advent of the Heliophysics/Geospace System Observatory (H/GSO), a complement of multi-spacecraft missions and ground-based observatories to study the space environment, data retrieval, analysis, and visualization of space physics data can be daunting. The Space Physics Environment Data Analysis System (SPEDAS), a grass-roots software development platform (www.spedas.org), is now officially supported by NASA Heliophysics as part of its data environment infrastructure. It serves more than a dozen space missions and ground observatories and can integrate the full complement of past and upcoming space physics missions with minimal resources, following clear, simple, and well-proven guidelines. Free, modular and configurable to the needs of individual missions, it works in both command-line (ideal for experienced users) and Graphical User Interface (GUI) mode (reducing the learning curve for first-time users). Both options have “crib-sheets,” user-command sequences in ASCII format that can facilitate record-and-repeat actions, especially for complex operations and plotting. Crib-sheets enhance scientific interactions, as users can move rapidly and accurately from exchanges of technical information on data processing to efficient discussions regarding data interpretation and science. SPEDAS can readily query and ingest all International Solar Terrestrial Physics (ISTP)-compatible products from the Space Physics Data Facility (SPDF), enabling access to a vast collection of historic and current mission data. The planned incorporation of Heliophysics Application Programmer’s Interface (HAPI) standards will facilitate data ingestion from distributed datasets that adhere to these standards. Although SPEDAS is currently Interactive Data Language (IDL)-based (and interfaces to Java-based tools such as Autoplot), efforts are under-way to expand it further to work with python (first as an interface tool and potentially even receiving an under-the-hood replacement). We review the SPEDAS development history, goals, and current implementation. We explain its “modes of use” with examples geared for users and outline its technical implementation and requirements with software developers in mind. We also describe SPEDAS personnel and software management, interfaces with other organizations, resources and support structure available to the community, and future development plans.Published versio