On the cost of composing shared-memory algorithms

Decades of research in distributed computing have led to a variety of perspectives on what it means for a concurrent algorithm to be efficient, depending on model assumptions, progress guarantees, and complexity metrics. It is therefore natural to ask whether one could compose algorithms that perform efficiently under different conditions, so that the composition preserves the performance of the original components when their conditions are met. In this paper, we evaluate the cost of composing shared-memory algorithms. First, we formally define the notion of safely composable algorithms and we show that every sequential type has a safely composable implementation, as long as enough state is transferred between modules. Since such generic implementations are inherently expensive, we present a more general light-weight specification that allows the designer to transfer very little state between modules, by taking advantage of the semantics of the implemented object. Using this framework, we implement a composed long-lived test-and-set object, with the property that each of its modules is asymptotically optimal with respect to the progress condition it ensures, while the entire implementation only uses objects with consensus number at most two. Thus, we show that the overhead of composition can be negligible in the case of some important shared-memory abstractions

The Complexity of Obstruction-Free Implementations

Obstruction-free implementations of concurrent ob jects are optimized for the common case where there is no step contention, and were recently advocated as a solution to the costs associated with synchronization without locks. In this paper, we study this claim and this goes through precisely defining the notions of obstruction-freedom and step contention. We consider several classes of obstruction-free implementations, present corresponding generic ob ject implementations, and prove lower bounds on their complexity. Viewed collectively, our results establish that the worst- case operation time complexity of obstruction-free implementations is high, of step contention. We also show that lock-based implementations are not sub ject to some of the time-complexity lower bounds we present

Notes on Theory of Distributed Systems

Notes for the Yale course CPSC 465/565 Theory of Distributed Systems

Partial Aborts for Transactions via First Class Continuations

Software transactional memory (STM) has proven to be a useful abstraction for developing concurrent applications where programmers denote transactions with an atomic construct that delimits a collection of reads and writes to shared mutable references. The runtime system then guarantees that all transactions are observed to execute atomically with respect to each other. Traditionally, when the runtime system detects that one transaction conflicts with another, it aborts one of the transactions and restarts its execution from the beginning. This can lead to problems with both execution time and throughput. This thesis presents a novel approach that uses first-class continuations to restart a conflicting transaction at the point of a conflict, avoiding the re-execution of any work from the beginning of the transaction that has not been compromised. In practice, this allows transactions to complete more quickly, decreasing execution time and increasing throughput. The ideas presented in this thesis have been implemented in the context of the Manticore project, an ML-family language with support for parallelism and concurrency. Crucially, this work relies on constant-time continuation capturing via a continuation-passing-style (CPS) transformation and heap-allocated continuations. The partial abort scheme has been implemented as a part of three modern STM implementations: TL2, TinySTM, and NOrec. Experimental results show that, while no base STM implementation is universally best, each partial-abort implementation compares favorably to its full-abort counterpart. In addition to an implementation, this thesis presents a formal semantics for partial aborts. A proof of correctness is given by relating the partial-abort semantics to an analogous full-abort semantics via a simulation. All proofs have been formally verified using the Coq Theorem Prover

Centralised monitoring and alerting solution for complex information management infrastructure

Internship Report presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced AnalyticsMonitoring and alerting receive increasing attention over the last years together with growth of the information generated and managed from one side and increasing of the computing power and capacity of the computing hardware. Monitoring solutions have been tightly linked to the software being monitored making the broad view of the performance of the all interlinked services too complex and ineffective. In this work a monitoring and alerting solution based on “Prometheus” is developed. Continuous collection of metrics from various different services is performed and organized for visualization and monitoring in several layers of precision. Different exporters for various systems were evaluated and many of them enhanced. Alerting logic in response of detected performance problems and function irregularities has been developed and implemented with “Alert Manager”. Another software is developed for recording and visualizing current or past alerts and also is used as debugging tool of the alerts configuration. Visualization is implemented for Grafana with several dashboards. All tools and software packages used for implementing this monitoring and alerting solution are open source and free to use