388 research outputs found
Merlin: A Language for Provisioning Network Resources
This paper presents Merlin, a new framework for managing resources in
software-defined networks. With Merlin, administrators express high-level
policies using programs in a declarative language. The language includes
logical predicates to identify sets of packets, regular expressions to encode
forwarding paths, and arithmetic formulas to specify bandwidth constraints. The
Merlin compiler uses a combination of advanced techniques to translate these
policies into code that can be executed on network elements including a
constraint solver that allocates bandwidth using parameterizable heuristics. To
facilitate dynamic adaptation, Merlin provides mechanisms for delegating
control of sub-policies and for verifying that modifications made to
sub-policies do not violate global constraints. Experiments demonstrate the
expressiveness and scalability of Merlin on real-world topologies and
applications. Overall, Merlin simplifies network administration by providing
high-level abstractions for specifying network policies and scalable
infrastructure for enforcing them
Toward High-Performance Computing and Big Data Analytics Convergence: The Case of Spark-DIY
Convergence between high-performance computing (HPC) and big data analytics (BDA) is currently an established research area that has spawned new opportunities for unifying the platform layer and data abstractions in these ecosystems. This work presents an architectural model that enables the interoperability of established BDA and HPC execution models, reflecting the key design features that interest both the HPC and BDA communities, and including an abstract data collection and operational model that generates a unified interface for hybrid applications. This architecture can be implemented in different ways depending on the process- and data-centric platforms of choice and the mechanisms put in place to effectively meet the requirements of the architecture. The Spark-DIY platform is introduced in the paper as a prototype implementation of the architecture proposed. It preserves the interfaces and execution environment of the popular BDA platform Apache Spark, making it compatible with any Spark-based application and tool, while providing efficient communication and kernel execution via DIY, a powerful communication pattern library built on top of MPI. Later, Spark-DIY is analyzed in terms of performance by building a representative use case from the hydrogeology domain, EnKF-HGS. This application is a clear example of how current HPC simulations are evolving toward hybrid HPC-BDA applications, integrating HPC simulations within a BDA environment.This work was supported in part by the Spanish Ministry of Economy, Industry and Competitiveness under Grant TIN2016-79637-P(toward Unification of HPC and Big Data Paradigms), in part by the Spanish Ministry of Education under Grant FPU15/00422 TrainingProgram for Academic and Teaching Staff Grant, in part by the Advanced Scientific Computing Research, Office of Science, U.S.Department of Energy, under Contract DE-AC02-06CH11357, and in part by the DOE with under Agreement DE-DC000122495,Program Manager Laura Biven
Autoscaling Hadoop Clusters
Pilve arvutused on viimaste aastate jooksul palju kõneainet pakkunud. Alates sellest, et
tegemist ei ole millegi muuga kui virtualiseerimine ilusa nimega, kuni selleni, et tulevik
on pilve arvutuste p aralt. Juba 4 aastat on virtuaalsed serverid, andmehoidlad, andmebaasid
ja muud infrastruktuuri elemendid olnud k attesaadavad veebiteenustena.
Antud töös me ehitame ise sklaleeruva MapReduce platvormi, mis baseerub vabalähtekoodiga
tarkvara Apache Hadoop projektil. Antud platvorm skaleerib end ise, vastavalt serverite
koormatusele k aivitab uusi servereid, et kiirendada arvutusprotsessi.Cloud computing, specifically Infrastructure as a Service model
provides us with the facilities to provision new servers at will and increase
the computing power of a cluster almost in real time. This provisioning
and deprovisioning of servers can happen automatically based on some
performance metrics of the cluster. We introduce a framework of autoscaling clusters in the private and public cloud ecosystem using the Eucalyptus and AWS
software stack and use MapReduce as the service provided by the cluster
- …