Transferring big data across the globe

Transmitting data via the Internet is a routine and common task for users today. The amount of data being transmitted by the average user has dramatically increased over the past few years. Transferring a gigabyte of data in an entire day was normal, however users are now transmitting multiple gigabytes in a single hour. With the influx of big data and massive scientific data sets that are measured in tens of petabytes, a user has the propensity to transfer even larger amounts of data. When transferring data sets of this magnitude on public or shared networks, the performance of all workloads in the system will be impacted. This dissertation addresses the issues and challenges inherent with transferring big data over shared networks. A survey of current transfer techniques is provided and these techniques are evaluated in simulated, experimental and live environments. The main contribution of this dissertation is the development of a new, nice model for big data transfers, which is based on a store-and-forward methodology instead of an end-to-end approach. This nice model ensures that big data transfers only occur when there is idle bandwidth that can be repurposed for these large transfers. The nice model improves overall performance and significantly reduces the transmission time for big data transfers. The model allows for efficient transfers regardless of time zone differences or variations in bandwidth between sender and receiver. Nice is the first model that addresses the challenges of transferring big data across the globe

Efficiently Clustering Very Large Attributed Graphs

Attributed graphs model real networks by enriching their nodes with attributes accounting for properties. Several techniques have been proposed for partitioning these graphs into clusters that are homogeneous with respect to both semantic attributes and to the structure of the graph. However, time and space complexities of state of the art algorithms limit their scalability to medium-sized graphs. We propose SToC (for Semantic-Topological Clustering), a fast and scalable algorithm for partitioning large attributed graphs. The approach is robust, being compatible both with categorical and with quantitative attributes, and it is tailorable, allowing the user to weight the semantic and topological components. Further, the approach does not require the user to guess in advance the number of clusters. SToC relies on well known approximation techniques such as bottom-k sketches, traditional graph-theoretic concepts, and a new perspective on the composition of heterogeneous distance measures. Experimental results demonstrate its ability to efficiently compute high-quality partitions of large scale attributed graphs.Comment: This work has been published in ASONAM 2017. This version includes an appendix with validation of our attribute model and distance function, omitted in the converence version for lack of space. Please refer to the published versio

Harmonizing Software Standards with a Semantic Model

The application of standards in the software development process supports interoperability between systems. Maintenance of standards must be guaranteed on the organisational and technical level. The use of semantic technologies can contribute to the standard maintenance process by providing a harmonizing bridge between standards of different knowledge domains and languages and by providing a single point of administration for standard domain concepts. This paper describes a case study of the creation of a semantic layer between software standards for water management systems in The Netherland

Tilting mutation of weakly symmetric algebras and stable equivalence

We consider tilting mutations of a weakly symmetric algebra at a subset of simple modules, as recently introduced by T. Aihara. These mutations are defined as the endomorphism rings of certain tilting complexes of length 1. Starting from a weakly symmetric algebra A, presented by a quiver with relations, we give a detailed description of the quiver and relations of the algebra obtained by mutating at a single loopless vertex of the quiver of A. In this form the mutation procedure appears similar to, although significantly more complicated than, the mutation procedure of Derksen, Weyman and Zelevinsky for quivers with potentials. By definition, weakly symmetric algebras connected by a sequence of tilting mutations are derived equivalent, and hence stably equivalent. The second aim of this article is to study these stable equivalences via a result of Okuyama describing the images of the simple modules. As an application we answer a question of Asashiba on the derived Picard groups of a class of self-injective algebras of finite representation type. We conclude by introducing a mutation procedure for maximal systems of orthogonal bricks in a triangulated category, which is motivated by the effect that a tilting mutation has on the set of simple modules in the stable category.Comment: Description and proof of mutated algebra made more rigorous (Prop. 3.1 and 4.2). Okuyama's Lemma incorporated: Theorem 4.1 is now Corollary 5.1, and proof is omitted. To appear in Algebras and Representation Theor

Aggregation of soil and climate input data can underestimate simulated biomass loss and nitrate leaching under climate change

Predicting areas of severe biomass loss and increased N leaching risk under climate change is critical for applying appropriate adaptation measures to support more sustainable agricultural systems. The frequency of annual severe biomass loss for winter wheat and its coincidence with an increase in N leaching in a temperate region in Germany was estimated including the error from using soil and climate input data at coarser spatial scales, using the soil-crop model CoupModel. We ran the model for a reference period (1980-2010) and used climate data predicted by four climate model(s) for the Representative Concentration Pathways (RCP) 2.6, 4.5 and 8.5. The annual median biomass estimations showed that for the period 2070-2100, under the RCP8.5 scenario, the entire region would suffer from severe biomass loss almost every year. Annual incidence of severe biomass loss and increased N leaching was predicted to increase from RCP4.5 to the 8.5 scenario. During 2070-2100 for RCP8.5, in more than half of the years an area of 95% of the region was projected to suffer from both severe biomass loss and increased N leaching. The SPEI3 predicted a range of 32 (P3 RCP4.5) to 55% (P3 RCP8.5) of the severe biomass loss episodes simulated in the climate change scenarios. The simulations predicted more severe biomass losses than by the SPEI index which indicates that soil water deficits are important in determining crop losses in future climate scenarios. There was a risk of overestimating the area where "no severe biomass loss + increased N leaching" occurred when using coarser aggregated input data. In contrast, underestimation of situations where "severe biomass loss + increased N leaching" occurred when using coarser aggregated input data. Larger annual differences in biomass estimations compared to the finest resolution of input data occurred when aggregating climate input data rather than soil data. The differences were even larger when aggregating both soil and climate input data. In half of the region, biomass could be erroneously estimated in a single year by more than 40% if using soil and climate coarser input data. The results suggest that a higher spatial resolution of especially climate input data would be needed to predict reliably annual estimates of severe biomass loss and N leaching under climate change scenarios

Aggregation of soil and climate input data can underestimate simulated biomass loss and nitrate leaching under climate change

Predicting areas of severe biomass loss and increased N leaching risk under climate change is critical for applying appropriate adaptation measures to support more sustainable agricultural systems. The frequency of annual severe biomass loss for winter wheat and its coincidence with an increase in N leaching in a temperate region in Germany was estimated including the error from using soil and climate input data at coarser spatial scales, using the soil-crop model CoupModel. We ran the model for a reference period (1980–2010) and used climate data predicted by four climate model(s) for the Representative Concentration Pathways (RCP) 2.6, 4.5 and 8.5. The annual median biomass estimations showed that for the period 2070–2100, under the RCP8.5 scenario, the entire region would suffer from severe biomass loss almost every year. Annual incidence of severe biomass loss and increased N leaching was predicted to increase from RCP4.5 to the 8.5 scenario. During 2070–2100 for RCP8.5, in more than half of the years an area of 95% of the region was projected to suffer from both severe biomass loss and increased N leaching. The SPEI3 predicted a range of 32 (P3 RCP4.5) to 55% (P3 RCP8.5) of the severe biomass loss episodes simulated in the climate change scenarios. The simulations predicted more severe biomass losses than by the SPEI index which indicates that soil water deficits are important in determining crop losses in future climate scenarios. There was a risk of overestimating the area where “no severe biomass loss + increased N leaching” occurred when using coarser aggregated input data. In contrast, underestimation of situations where “severe biomass loss + increased N leaching” occurred when using coarser aggregated input data. Larger annual differences in biomass estimations compared to the finest resolution of input data occurred when aggregating climate input data rather than soil data. The differences were even larger when aggregating both soil and climate input data. In half of the region, biomass could be erroneously estimated in a single year by more than 40% if using soil and climate coarser input data. The results suggest that a higher spatial resolution of especially climate input data would be needed to predict reliably annual estimates of severe biomass loss and N leaching under climate change scenarios.Peer reviewe

Overfitting control inside cascade correlation neural networks applied to electricity contract price prediction

Prediction of electricity prices is considered a difficult task due to the number and complexity of factors that influence their performance, and their relationships. Neural networks cascade correlation - CASCOR allows to do a constructive learning and it captures better the characteristics of the data; however, it has a high tendency to overfitting. To control overfitting in some areas regularization techniques are used. However, in the literature there are no studies that: i) use regularization techniques to control overfitting in CASCOR networks, ii) use CASCOR networks in predicting of electrical series iii) compare the performance with tra­ditional neural networks or statistical models. The aim of this paper is to model and predict the behavior of the price series of electricity contracts in Colombia, using CASCOR networks and controlling the overfitting by regularization techniques.La predicción de precios de electricidad es considerada una tarea difí­cil debido a la cantidad y complejidad de los factores que influyen en su representación, y sus relaciones. Las redes neuronales tipo cascada correlación –CASCOR– permiten, realizar un aprendizaje constructivo, capturando mejor las características de los datos; sin embargo, presentan una alta tendencia al sobreajuste. Para el control del sobreajuste en algunos ámbitos se usan técnicas de regularización. No obstante, en la literatura no existen estudios que: i) Utilicen técnicas de regularización para el control de sobreajuste en redes CASCOR; ii) Usen redes CASCOR en la predicción de series de electricidad; iii) comparen el desempeño con redes neuronales tradicionales o modelos estadísticos. El objetivo de este artículo es modelar y predecir el comportamiento de la serie de precios de contratos de electricidad en Colombia, usando redes CASCOR y con­trolando el sobreajuste con técnicas de regularización