Scalable Persistent Storage for Erlang

The many core revolution makes scalability a key property. The RELEASE project aims to improve the scalability of Erlang on emergent commodity architectures with 100,000 cores. Such architectures require scalable and available persistent storage on up to 100 hosts. We enumerate the requirements for scalable and available persistent storage, and evaluate four popular Erlang DBMSs against these requirements. This analysis shows that Mnesia and CouchDB are not suitable persistent storage at our target scale, but Dynamo-like NoSQL DataBase Management Systems (DBMSs) such as Cassandra and Riak potentially are. We investigate the current scalability limits of the Riak 1.1.1 NoSQL DBMS in practice on a 100-node cluster. We establish for the first time scientifically the scalability limit of Riak as 60 nodes on the Kalkyl cluster, thereby confirming developer folklore. We show that resources like memory, disk, and network do not limit the scalability of Riak. By instrumenting Erlang/OTP and Riak libraries we identify a specific Riak functionality that limits scalability. We outline how later releases of Riak are refactored to eliminate the scalability bottlenecks. We conclude that Dynamo-style NoSQL DBMSs provide scalable and available persistent storage for Erlang in general, and for our RELEASE target architecture in particular

RELEASE: A High-level Paradigm for Reliable Large-scale Server Software

Erlang is a functional language with a much-emulated model for building reliable distributed systems. This paper outlines the RELEASE project, and describes the progress in the first six months. The project aim is to scale the Erlang’s radical concurrency-oriented programming paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines. Currently Erlang has inherently scalable computation and reliability models, but in practice scalability is constrained by aspects of the language and virtual machine. We are working at three levels to address these challenges: evolving the Erlang virtual machine so that it can work effectively on large scale multicore systems; evolving the language to Scalable Distributed (SD) Erlang; developing a scalable Erlang infrastructure to integrate multiple, heterogeneous clusters. We are also developing state of the art tools that allow programmers to understand the behaviour of massively parallel SD Erlang programs. We will demonstrate the effectiveness of the RELEASE approach using demonstrators and two large case studies on a Blue Gene

RELEASE: A High-level Paradigm for Reliable Large-scale Server Software

Erlang is a functional language with a much-emulated model for building reliable distributed systems. This paper outlines the RELEASE project, and describes the progress in the rst six months. The project aim is to scale the Erlang's radical concurrency-oriented programming paradigm to build reliable general-purpose software, such as server-based systems, on massively parallel machines. Currently Erlang has inherently scalable computation and reliability models, but in practice scalability is constrained by aspects of the language and virtual machine. We are working at three levels to address these challenges: evolving the Erlang virtual machine so that it can work effectively on large scale multicore systems; evolving the language to Scalable Distributed (SD) Erlang; developing a scalable Erlang infrastructure to integrate multiple, heterogeneous clusters. We are also developing state of the art tools that allow programmers to understand the behaviour of massively parallel SD Erlang programs. We will demonstrate the e ectiveness of the RELEASE approach using demonstrators and two large case studies on a Blue Gene

An Analysis of Scripting Languages for Research in Applied Computing

There are several scripting languages that exist today. However, some are used more frequently and popular than others. This is due to certain characteristics and features that they possess. Particularly in applied computing fields like software engineering, bioinformatics and computational biology, scripting languages are gaining popularity. This paper presents a comparative study of ten popular scripting languages that are used in the above mentioned fields/area. For making comparison, we have identified the factors against which these languages are evaluated. Accordingly, based on selected criteria we determine their suitability in the fields of software engineering, bioinformatics and computational biology research. This will serve as a guide to researchers to choose the appropriate scripting language in the various fields

Bad Luck When Joining the Shortest Queue

A frequent observation in service systems with queues in parallel is that customers in other queues tend to be served faster than those in one’s own queue. This paper quantifies the probability that one’s service would have started earlier if one had joined another queue than the queue that was actually chosen, for exponential multiserver systems with queues in parallel in which customers join one of the shortest queues upon arrival and in which jockeying is not possible.Queueing;Join-the-shortest-queue;Probability of bad luck;Power-series algorithm;Overtaking customers;Dedicated customers

Generation-free Agent-based Evolutionary Computing

AbstractMetaheuristics resulting from the hybridization of multi-agent systems with evolutionary computing are efficient in many optimization problems. Evolutionary multi-agent systems (EMAS) are more similar to biological evolution than classical evolutionary algorithms. However, technological limitations prevented the use of fully asynchronous agents in previous EMAS implementations. In this paper we present a new algorithm for agent-based evolutionary computations. The individuals are represented as fully autonomous and asynchronous agents. Evolutionary operations are performed continuously and no artificial generations need to be distinguished. Our results show that such asynchronous evolutionary operators and the resulting absence of explicit generations lead to significantly better results. An efficient implementation of this algorithm was possible through the use of Erlang technology, which natively supports lightweight processes and asynchronous communication