258 research outputs found
Towards Python-based Domain-specific Languages for Self-reconfigurable Modular Robotics Research
This paper explores the role of operating system and high-level languages in
the development of software and domain-specific languages (DSLs) for
self-reconfigurable robotics. We review some of the current trends in
self-reconfigurable robotics and describe the development of a software system
for ATRON II which utilizes Linux and Python to significantly improve software
abstraction and portability while providing some basic features which could
prove useful when using Python, either stand-alone or via a DSL, on a
self-reconfigurable robot system. These features include transparent socket
communication, module identification, easy software transfer and reliable
module-to-module communication. The end result is a software platform for
modular robots that where appropriate builds on existing work in operating
systems, virtual machines, middleware and high-level languages.Comment: Presented at DSLRob 2011 (arXiv:1212.3308
numpywren: serverless linear algebra
Linear algebra operations are widely used in scientific computing and machine
learning applications. However, it is challenging for scientists and data
analysts to run linear algebra at scales beyond a single machine. Traditional
approaches either require access to supercomputing clusters, or impose
configuration and cluster management challenges. In this paper we show how the
disaggregation of storage and compute resources in so-called "serverless"
environments, combined with compute-intensive workload characteristics, can be
exploited to achieve elastic scalability and ease of management.
We present numpywren, a system for linear algebra built on a serverless
architecture. We also introduce LAmbdaPACK, a domain-specific language designed
to implement highly parallel linear algebra algorithms in a serverless setting.
We show that, for certain linear algebra algorithms such as matrix multiply,
singular value decomposition, and Cholesky decomposition, numpywren's
performance (completion time) is within 33% of ScaLAPACK, and its compute
efficiency (total CPU-hours) is up to 240% better due to elasticity, while
providing an easier to use interface and better fault tolerance. At the same
time, we show that the inability of serverless runtimes to exploit locality
across the cores in a machine fundamentally limits their network efficiency,
which limits performance on other algorithms such as QR factorization. This
highlights how cloud providers could better support these types of computations
through small changes in their infrastructure
Recommended from our members
FabSim3: An automation toolkit for verified simulations using high performance computing
A common feature of computational modelling and simulation research is the need to perform many
tasks in complex sequences to achieve a usable result. This will typically involve tasks such as preparing
input data, pre-processing, running simulations on a local or remote machine, post-processing, and
performing coupling communications, validations and/or optimisations. Tasks like these can involve
manual steps which are time and effort intensive, especially when it involves the management of large
ensemble runs. Additionally, human errors become more likely and numerous as the research work
becomes more complex, increasing the risk of damaging the credibility of simulation results. Automation
tools can help ensure the credibility of simulation results by reducing the manual time and effort
required to perform these research tasks, by making more rigorous procedures tractable, and by reducing
the probability of human error due to a reduced number of manual actions. In addition, efficiency
gained through automation can help researchers to perform more research within the budget and effort
constraints imposed by their projects.
This paper presents the main software release of FabSim3, and explains how our automation toolkit
can improve and simplify a range of tasks for researchers and application developers. FabSim3 helps
to prepare, submit, execute, retrieve, and analyze simulation workflows. By providing a suitable level
of abstraction, FabSim3 reduces the complexity of setting up and managing a large-scale simulation
scenario, while still providing transparent access to the underlying layers for effective debugging.
The tool also facilitates job submission and management (including staging and curation of files
and environments) for a range of different supercomputing environments. Although FabSim3 itself is
application-agnostic, it supports a provably extensible plugin system where users automate simulation
and analysis workflows for their own application domains. To highlight this, we briefly describe a
selection of these plugins and we demonstrate the efficiency of the toolkit in handling large ensemble
workflows.EPSRC under grant agreement EP/W007711/1, as well as by the VECMA and HiDALGO projects, which have
received funding from the European Union Horizon 2020 research and innovation programme under grant agreement nos 800925 and
824115. In addition, FabFlee was supported by the ITFLOWS project and FabCovid19 by the STAMINA project, both of which have received
funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 882986 and No 883441
respectivel
- …