Data intensive scientific analysis with grid computing

At the end of September 2009, a new Italian GPS receiver for radio occultation was launched from the Satish Dhawan Space Center (Sriharikota, India) on the Indian Remote Sensing OCEANSAT-2 satellite. The Italian Space Agency has established a set of Italian universities and research centers to implement the overall processing radio occultation chain. After a brief description of the adopted algorithms, which can be used to characterize the temperature, pressure and humidity, the contribution will focus on a method for automatic processing these data, based on the use of a distributed architecture. This paper aims at being a possible application of grid computing for scientific research

Enhancing Job Scheduling of an Atmospheric Intensive Data Application

Nowadays, e-Science applications involve great deal of data to have more accurate analysis. One of its application domains is the Radio Occultation which manages satellite data. Grid Processing Management is a physical infrastructure geographically distributed based on Grid Computing, that is implemented for the overall processing Radio Occultation analysis. After a brief description of algorithms adopted to characterize atmospheric profiles, the paper presents an improvement of job scheduling in order to decrease processing time and optimize resource utilization. Extension of grid computing capacity is implemented by virtual machines in existing physical Grid in order to satisfy temporary job requests. Also scheduling plays an important role in the infrastructure that is handled by a couple of schedulers which are developed to manage data automaticall

An Integrated Approach for Characterizing Aerosol Climate Impacts and Environmental Interactions

Aerosols exert myriad influences on the earth's environment and climate, and on human health. The complexity of aerosol-related processes requires that information gathered to improve our understanding of climate change must originate from multiple sources, and that effective strategies for data integration need to be established. While a vast array of observed and modeled data are becoming available, the aerosol research community currently lacks the necessary tools and infrastructure to reap maximum scientific benefit from these data. Spatial and temporal sampling differences among a diverse set of sensors, nonuniform data qualities, aerosol mesoscale variabilities, and difficulties in separating cloud effects are some of the challenges that need to be addressed. Maximizing the long-term benefit from these data also requires maintaining consistently well-understood accuracies as measurement approaches evolve and improve. Achieving a comprehensive understanding of how aerosol physical, chemical, and radiative processes impact the earth system can be achieved only through a multidisciplinary, inter-agency, and international initiative capable of dealing with these issues. A systematic approach, capitalizing on modern measurement and modeling techniques, geospatial statistics methodologies, and high-performance information technologies, can provide the necessary machinery to support this objective. We outline a framework for integrating and interpreting observations and models, and establishing an accurate, consistent, and cohesive long-term record, following a strategy whereby information and tools of progressively greater sophistication are incorporated as problems of increasing complexity are tackled. This concept is named the Progressive Aerosol Retrieval and Assimilation Global Observing Network (PARAGON). To encompass the breadth of the effort required, we present a set of recommendations dealing with data interoperability; measurement and model integration; multisensor synergy; data summarization and mining; model evaluation; calibration and validation; augmentation of surface and in situ measurements; advances in passive and active remote sensing; and design of satellite missions. Without an initiative of this nature, the scientific and policy communities will continue to struggle with understanding the quantitative impact of complex aerosol processes on regional and global climate change and air quality

How organized is deep convection over Germany?

Deep moist convection shows a tendency to organize into mesoscale structures. To be able to understand the potential effect of convective organization on the climate, one needs first to characterize organization. In this study, we systematically characterize the organizational state of convection over Germany based on two years of cloud-top observations derived from the Meteosat Second Generation satellite and of precipitation cores detected by the German C-band radar network. The organizational state of convection is characterized by commonly employed organization indices, which are mostly based on the object numbers, sizes and nearest-neighbour distances. According to the organization index Iorg, cloud tops and precipitation cores are found to be in an organized state for 69% and 92% of the time, respectively. There is an increase in rainfall when the number of objects and their sizes increase, independently of the organizational state. Case-studies of specific days suggest that convectively organized states correspond to either local multi-cell clusters, with less numerous, larger objects close to each other, or to scattered clusters, with more numerous, smaller organized objects spread out over the domain. For those days, simulations are performed with the large-eddy model ICON with grid spacings of 625, 312 and 156?m. Although the model underestimates rainfall and shows a too large cold cloud coverage, the organizational state is reasonably well represented without significant differences between the grid spacings

A Numerical Study of Methods for Moist Atmospheric Flows: Compressible Equations

We investigate two common numerical techniques for integrating reversible moist processes in atmospheric flows in the context of solving the fully compressible Euler equations. The first is a one-step, coupled technique based on using appropriate invariant variables such that terms resulting from phase change are eliminated in the governing equations. In the second approach, which is a two-step scheme, separate transport equations for liquid water and vapor water are used, and no conversion between water vapor and liquid water is allowed in the first step, while in the second step a saturation adjustment procedure is performed that correctly allocates the water into its two phases based on the Clausius-Clapeyron formula. The numerical techniques we describe are first validated by comparing to a well-established benchmark problem. Particular attention is then paid to the effect of changing the time scale at which the moist variables are adjusted to the saturation requirements in two different variations of the two-step scheme. This study is motivated by the fact that when acoustic modes are integrated separately in time (neglecting phase change related phenomena), or when sound-proof equations are integrated, the time scale for imposing saturation adjustment is typically much larger than the numerical one related to the acoustics

Vertical wind profile characterization and identification of patterns based on a shape clustering algorithm

Wind power plants are becoming a generally accepted resource in the generation mix of many utilities. At the same time, the size and the power rating of individual wind turbines have increased considerably. Under these circumstances, the sector is increasingly demanding an accurate characterization of vertical wind speed profiles to estimate properly the incoming wind speed at the rotor swept area and, consequently, assess the potential for a wind power plant site. The present paper describes a shape-based clustering characterization and visualization of real vertical wind speed data. The proposed solution allows us to identify the most likely vertical wind speed patterns for a specific location based on real wind speed measurements. Moreover, this clustering approach also provides characterization and classification of such vertical wind profiles. This solution is highly suitable for a large amount of data collected by remote sensing equipment, where wind speed values at different heights within the rotor swept area are available for subsequent analysis. The methodology is based on z-normalization, shape-based distance metric solution and the Ward-hierarchical clustering method. Real vertical wind speed profile data corresponding to a Spanish wind power plant and collected by using a commercialWindcube equipment during several months are used to assess the proposed characterization and clustering process, involving more than 100000 wind speed data values. All analyses have been implemented using open-source R-software. From the results, at least four different vertical wind speed patterns are identified to characterize properly over 90% of the collected wind speed data along the day. Therefore, alternative analytical function criteria should be subsequently proposed for vertical wind speed characterization purposes.The authors are grateful for the financial support from the Spanish Ministry of the Economy and Competitiveness and the European Union —ENE2016-78214-C2-2-R—and the Spanish Education, Culture and Sport Ministry —FPU16/042

A Practical Method to Estimate Information Content in the Context of 4D-Var Data Assimilation. II: Application to Global Ozone Assimilation

Data assimilation obtains improved estimates of the state of a physical system by combining imperfect model results with sparse and noisy observations of reality. Not all observations used in data assimilation are equally valuable. The ability to characterize the usefulness of different data points is important for analyzing the effectiveness of the assimilation system, for data pruning, and for the design of future sensor systems. In the companion paper (Sandu et al., 2012) we derive an ensemble-based computational procedure to estimate the information content of various observations in the context of 4D-Var. Here we apply this methodology to quantify the signal and degrees of freedom for signal information metrics of satellite observations used in a global chemical data assimilation problem with the GEOS-Chem chemical transport model. The assimilation of a subset of data points characterized by the highest information content yields an analysis comparable in quality with the one obtained using the entire data set