11,227 research outputs found
Transferring big data across the globe
Transmitting data via the Internet is a routine and common task for users today. The amount of data being transmitted by the average user has dramatically increased over the past few years. Transferring a gigabyte of data in an entire day was normal, however users are now transmitting multiple gigabytes in a single hour. With the influx of big data and massive scientific data sets that are measured in tens of petabytes, a user has the propensity to transfer even larger amounts of data. When transferring data sets of this magnitude on public or shared networks, the performance of all workloads in the system will be impacted.
This dissertation addresses the issues and challenges inherent with transferring big data over shared networks. A survey of current transfer techniques is provided and these techniques are evaluated in simulated, experimental and live environments. The main contribution of this dissertation is the development of a new, nice model for big data transfers, which is based on a store-and-forward methodology instead of an end-to-end approach. This nice model ensures that big data transfers only occur when there is idle bandwidth that can be repurposed for these large transfers. The nice model improves overall performance and significantly reduces the transmission time for big data transfers. The model allows for efficient transfers regardless of time zone differences or variations in bandwidth between sender and receiver. Nice is the first model that addresses the challenges of transferring big data across the globe
Efficiently Clustering Very Large Attributed Graphs
Attributed graphs model real networks by enriching their nodes with
attributes accounting for properties. Several techniques have been proposed for
partitioning these graphs into clusters that are homogeneous with respect to
both semantic attributes and to the structure of the graph. However, time and
space complexities of state of the art algorithms limit their scalability to
medium-sized graphs. We propose SToC (for Semantic-Topological Clustering), a
fast and scalable algorithm for partitioning large attributed graphs. The
approach is robust, being compatible both with categorical and with
quantitative attributes, and it is tailorable, allowing the user to weight the
semantic and topological components. Further, the approach does not require the
user to guess in advance the number of clusters. SToC relies on well known
approximation techniques such as bottom-k sketches, traditional graph-theoretic
concepts, and a new perspective on the composition of heterogeneous distance
measures. Experimental results demonstrate its ability to efficiently compute
high-quality partitions of large scale attributed graphs.Comment: This work has been published in ASONAM 2017. This version includes an
appendix with validation of our attribute model and distance function,
omitted in the converence version for lack of space. Please refer to the
published versio
Harmonizing Software Standards with a Semantic Model
The application of standards in the software development process supports interoperability between systems. Maintenance of standards must be guaranteed on the organisational and technical level. The use of semantic technologies can contribute to the standard maintenance process by providing a harmonizing bridge between standards of different knowledge domains and languages and by providing a single point of administration for standard domain concepts. This paper describes a case study of the creation of a semantic layer between software standards for water management systems in The Netherland
Tilting mutation of weakly symmetric algebras and stable equivalence
We consider tilting mutations of a weakly symmetric algebra at a subset of
simple modules, as recently introduced by T. Aihara. These mutations are
defined as the endomorphism rings of certain tilting complexes of length 1.
Starting from a weakly symmetric algebra A, presented by a quiver with
relations, we give a detailed description of the quiver and relations of the
algebra obtained by mutating at a single loopless vertex of the quiver of A. In
this form the mutation procedure appears similar to, although significantly
more complicated than, the mutation procedure of Derksen, Weyman and Zelevinsky
for quivers with potentials. By definition, weakly symmetric algebras connected
by a sequence of tilting mutations are derived equivalent, and hence stably
equivalent. The second aim of this article is to study these stable
equivalences via a result of Okuyama describing the images of the simple
modules. As an application we answer a question of Asashiba on the derived
Picard groups of a class of self-injective algebras of finite representation
type. We conclude by introducing a mutation procedure for maximal systems of
orthogonal bricks in a triangulated category, which is motivated by the effect
that a tilting mutation has on the set of simple modules in the stable
category.Comment: Description and proof of mutated algebra made more rigorous (Prop.
3.1 and 4.2). Okuyama's Lemma incorporated: Theorem 4.1 is now Corollary 5.1,
and proof is omitted. To appear in Algebras and Representation Theor
Aggregation of soil and climate input data can underestimate simulated biomass loss and nitrate leaching under climate change
Predicting areas of severe biomass loss and increased N leaching risk under climate change is critical for applying appropriate adaptation measures to support more sustainable agricultural systems. The frequency of annual severe biomass loss for winter wheat and its coincidence with an increase in N leaching in a temperate region in Germany was estimated including the error from using soil and climate input data at coarser spatial scales, using the soil-crop model CoupModel. We ran the model for a reference period (1980-2010) and used climate data predicted by four climate model(s) for the Representative Concentration Pathways (RCP) 2.6, 4.5 and 8.5. The annual median biomass estimations showed that for the period 2070-2100, under the RCP8.5 scenario, the entire region would suffer from severe biomass loss almost every year. Annual incidence of severe biomass loss and increased N leaching was predicted to increase from RCP4.5 to the 8.5 scenario. During 2070-2100 for RCP8.5, in more than half of the years an area of 95% of the region was projected to suffer from both severe biomass loss and increased N leaching. The SPEI3 predicted a range of 32 (P3 RCP4.5) to 55% (P3 RCP8.5) of the severe biomass loss episodes simulated in the climate change scenarios. The simulations predicted more severe biomass losses than by the SPEI index which indicates that soil water deficits are important in determining crop losses in future climate scenarios. There was a risk of overestimating the area where "no severe biomass loss + increased N leaching" occurred when using coarser aggregated input data. In contrast, underestimation of situations where "severe biomass loss + increased N leaching" occurred when using coarser aggregated input data. Larger annual differences in biomass estimations compared to the finest resolution of input data occurred when aggregating climate input data rather than soil data. The differences were even larger when aggregating both soil and climate input data. In half of the region, biomass could be erroneously estimated in a single year by more than 40% if using soil and climate coarser input data. The results suggest that a higher spatial resolution of especially climate input data would be needed to predict reliably annual estimates of severe biomass loss and N leaching under climate change scenarios
Aggregation of soil and climate input data can underestimate simulated biomass loss and nitrate leaching under climate change
Predicting areas of severe biomass loss and increased N leaching risk under climate change is critical for applying appropriate adaptation measures to support more sustainable agricultural systems. The frequency of annual severe biomass loss for winter wheat and its coincidence with an increase in N leaching in a temperate region in Germany was estimated including the error from using soil and climate input data at coarser spatial scales, using the soil-crop model CoupModel. We ran the model for a reference period (1980–2010) and used climate data predicted by four climate model(s) for the Representative Concentration Pathways (RCP) 2.6, 4.5 and 8.5. The annual median biomass estimations showed that for the period 2070–2100, under the RCP8.5 scenario, the entire region would suffer from severe biomass loss almost every year. Annual incidence of severe biomass loss and increased N leaching was predicted to increase from RCP4.5 to the 8.5 scenario. During 2070–2100 for RCP8.5, in more than half of the years an area of 95% of the region was projected to suffer from both severe biomass loss and increased N leaching. The SPEI3 predicted a range of 32 (P3 RCP4.5) to 55% (P3 RCP8.5) of the severe biomass loss episodes simulated in the climate change scenarios. The simulations predicted more severe biomass losses than by the SPEI index which indicates that soil water deficits are important in determining crop losses in future climate scenarios. There was a risk of overestimating the area where “no severe biomass loss + increased N leaching” occurred when using coarser aggregated input data. In contrast, underestimation of situations where “severe biomass loss + increased N leaching” occurred when using coarser aggregated input data. Larger annual differences in biomass estimations compared to the finest resolution of input data occurred when aggregating climate input data rather than soil data. The differences were even larger when aggregating both soil and climate input data. In half of the region, biomass could be erroneously estimated in a single year by more than 40% if using soil and climate coarser input data. The results suggest that a higher spatial resolution of especially climate input data would be needed to predict reliably annual estimates of severe biomass loss and N leaching under climate change scenarios.Peer reviewe
Overfitting control inside cascade correlation neural networks applied to electricity contract price prediction
Prediction of electricity prices is considered a difficult task due to the number and complexity of factors that influence their performance, and their relationships. Neural networks cascade correlation - CASCOR allows to do a constructive learning and it captures better the characteristics of the data; however, it has a high tendency to overfitting. To control overfitting in some areas regularization techniques are used. However, in the literature there are no studies that: i) use regularization techniques to control overfitting in CASCOR networks, ii) use CASCOR networks in predicting of electrical series iii) compare the performance with traditional neural networks or statistical models. The aim of this paper is to model and predict the behavior of the price series of electricity contracts in Colombia, using CASCOR networks and controlling the overfitting by regularization techniques.La predicción de precios de electricidad es considerada una tarea difícil debido a la cantidad y complejidad de los factores que influyen en su representación, y sus relaciones. Las redes neuronales tipo cascada correlación –CASCOR– permiten, realizar un aprendizaje constructivo, capturando mejor las características de los datos; sin embargo, presentan una alta tendencia al sobreajuste. Para el control del sobreajuste en algunos ámbitos se usan técnicas de regularización. No obstante, en la literatura no existen estudios que: i) Utilicen técnicas de regularización para el control de sobreajuste en redes CASCOR; ii) Usen redes CASCOR en la predicción de series de electricidad; iii) comparen el desempeño con redes neuronales tradicionales o modelos estadísticos. El objetivo de este artículo es modelar y predecir el comportamiento de la serie de precios de contratos de electricidad en Colombia, usando redes CASCOR y controlando el sobreajuste con técnicas de regularización
- …