25,041 research outputs found
BaseFs - Basically Acailable, Soft State, Eventually Consistent Filesystem for Cluster Management
A peer-to-peer distributed filesystem for community cloud management. https://github.com/glic3rinu/basef
Integrated structural analysis tool using linear matching method part 1 : Software development
A number of direct methods based upon the Linear Matching Method (LMM) framework have been developed to address structural integrity issues for components subjected to cyclic thermal and mechanical load conditions. This paper presents a new integrated structural analysis tool using the LMM framework for the assessment of load carrying capacity, shakedown limit, ratchet limit and steady state cyclic response of structures. First, the development of the LMM for the evaluation of design limits in plasticity is introduced. Second, preliminary considerations for the development of the LMM into a tool which can be used on a regular basis by engineers are discussed. After the re-structuring of the LMM subroutines for multiple CPU solution, the LMM software tool for the assessment of design limits in plasticity is implemented by developing an Abaqus CAE plug-Âin with graphical user interfaces. Further demonstration of this new LMM analysis tool including practical application and verification is presented in an accompanying paper
DALiuGE: A Graph Execution Framework for Harnessing the Astronomical Data Deluge
The Data Activated Liu Graph Engine - DALiuGE - is an execution framework for
processing large astronomical datasets at a scale required by the Square
Kilometre Array Phase 1 (SKA1). It includes an interface for expressing complex
data reduction pipelines consisting of both data sets and algorithmic
components and an implementation run-time to execute such pipelines on
distributed resources. By mapping the logical view of a pipeline to its
physical realisation, DALiuGE separates the concerns of multiple stakeholders,
allowing them to collectively optimise large-scale data processing solutions in
a coherent manner. The execution in DALiuGE is data-activated, where each
individual data item autonomously triggers the processing on itself. Such
decentralisation also makes the execution framework very scalable and flexible,
supporting pipeline sizes ranging from less than ten tasks running on a laptop
to tens of millions of concurrent tasks on the second fastest supercomputer in
the world. DALiuGE has been used in production for reducing interferometry data
sets from the Karl E. Jansky Very Large Array and the Mingantu Ultrawide
Spectral Radioheliograph; and is being developed as the execution framework
prototype for the Science Data Processor (SDP) consortium of the Square
Kilometre Array (SKA) telescope. This paper presents a technical overview of
DALiuGE and discusses case studies from the CHILES and MUSER projects that use
DALiuGE to execute production pipelines. In a companion paper, we provide
in-depth analysis of DALiuGE's scalability to very large numbers of tasks on
two supercomputing facilities.Comment: 31 pages, 12 figures, currently under review by Astronomy and
Computin
A Random Structure for Optimum Cache Size Distributed hash table (DHT) Peer-to-Peer design
We propose a new and easily-realizable distributed hash table (DHT)
peer-to-peer structure, incorporating a random caching strategy that allows for
{\em polylogarithmic search time} while having only a {\em constant cache}
size. We also show that a very large class of deterministic caching strategies,
which covers almost all previously proposed DHT systems, can not achieve
polylog search time with constant cache size. In general, the new scheme is the
first known DHT structure with the following highly-desired properties: (a)
Random caching strategy with constant cache size; (b) Average search time of
; (c) Guaranteed search time of ; (d) Truly local
cache dynamics with constant overhead for node deletions and additions; (e)
Self-organization from any initial network state towards the desired structure;
and (f) Allows a seamless means for various trade-offs, e.g., search speed or
anonymity at the expense of larger cache size.Comment: 13 pages, 2 figures, preprint versio
A Tale of Two Data-Intensive Paradigms: Applications, Abstractions, and Architectures
Scientific problems that depend on processing large amounts of data require
overcoming challenges in multiple areas: managing large-scale data
distribution, co-placement and scheduling of data with compute resources, and
storing and transferring large volumes of data. We analyze the ecosystems of
the two prominent paradigms for data-intensive applications, hereafter referred
to as the high-performance computing and the Apache-Hadoop paradigm. We propose
a basis, common terminology and functional factors upon which to analyze the
two approaches of both paradigms. We discuss the concept of "Big Data Ogres"
and their facets as means of understanding and characterizing the most common
application workloads found across the two paradigms. We then discuss the
salient features of the two paradigms, and compare and contrast the two
approaches. Specifically, we examine common implementation/approaches of these
paradigms, shed light upon the reasons for their current "architecture" and
discuss some typical workloads that utilize them. In spite of the significant
software distinctions, we believe there is architectural similarity. We discuss
the potential integration of different implementations, across the different
levels and components. Our comparison progresses from a fully qualitative
examination of the two paradigms, to a semi-quantitative methodology. We use a
simple and broadly used Ogre (K-means clustering), characterize its performance
on a range of representative platforms, covering several implementations from
both paradigms. Our experiments provide an insight into the relative strengths
of the two paradigms. We propose that the set of Ogres will serve as a
benchmark to evaluate the two paradigms along different dimensions.Comment: 8 pages, 2 figure
Towards MKM in the Large: Modular Representation and Scalable Software Architecture
MKM has been defined as the quest for technologies to manage mathematical
knowledge. MKM "in the small" is well-studied, so the real problem is to scale
up to large, highly interconnected corpora: "MKM in the large". We contend that
advances in two areas are needed to reach this goal. We need representation
languages that support incremental processing of all primitive MKM operations,
and we need software architectures and implementations that implement these
operations scalably on large knowledge bases.
We present instances of both in this paper: the MMT framework for modular
theory-graphs that integrates meta-logical foundations, which forms the base of
the next OMDoc version; and TNTBase, a versioned storage system for XML-based
document formats. TNTBase becomes an MMT database by instantiating it with
special MKM operations for MMT.Comment: To appear in The 9th International Conference on Mathematical
Knowledge Management: MKM 201
A Pragmatic Approach to DHT Adoption
Despite the peer-to-peer community's obvious wish to have its systems adopted, specific mechanisms to facilitate incremental adoption have not yet received the same level of attention as the many other practical concerns associated with these systems. This paper argues that ease of adoption should be elevated to a first-class concern and accordingly presents HOLD, a front-end to existing DHTs that is optimized for incremental adoption. Specifically, HOLD is backwards-compatible: it leverages DNS to provide a key-based routing service to existing Internet hosts without requiring them to install any software. This paper also presents applications that could benefit from HOLD as well as the trade-offs that accompany HOLD. Early implementation experience suggests that HOLD is practical
Random Linear Network Coding for 5G Mobile Video Delivery
An exponential increase in mobile video delivery will continue with the
demand for higher resolution, multi-view and large-scale multicast video
services. Novel fifth generation (5G) 3GPP New Radio (NR) standard will bring a
number of new opportunities for optimizing video delivery across both 5G core
and radio access networks. One of the promising approaches for video quality
adaptation, throughput enhancement and erasure protection is the use of
packet-level random linear network coding (RLNC). In this review paper, we
discuss the integration of RLNC into the 5G NR standard, building upon the
ideas and opportunities identified in 4G LTE. We explicitly identify and
discuss in detail novel 5G NR features that provide support for RLNC-based
video delivery in 5G, thus pointing out to the promising avenues for future
research.Comment: Invited paper for Special Issue "Network and Rateless Coding for
Video Streaming" - MDPI Informatio
- …