Supporting persistent C++ objects in a distributed storage system

technical reportWe have designed and implemented a C++ object layer for Khazana, a distributed persistent storage system that exports a flat shared address space as its basic abstraction. The C++ layer described herein lets programmers use familiar C++ idioms to allocate, manipulate, and deallocate persistent shared data structures. It handles the tedious details involved in accessing this shared data, replicating it, maintaining consistency, converting data representations between persistent and in-memory representations, associating type information including methods with objects, etc. To support the C++ object layer on top of Khazana's flat storage abstraction, we have developed a language-specific preprocessor that generates support code to manage the user-specified persistent C++ structures. We describe the design of the C++ object layer and the compiler and runtime mechanisms needed to support it

Using khazana to support distributed application development

technical reportOne of the most important services required by most distributed applications is some form of shared data management, e.g., a directory service manages shared directory entries while groupware manages shared documents. Each such application currently must implement its own data management mechanisms, because existing runtime systems are not flexible enough to support all distributed applications efficiently. For example, groupware can be efficiently supported by a distributed object system, while a distributed database would prefer a more low-level storage abstraction. The goal of Khazana is to provide programmer's with configurable components that support the data management services required by a wide variety of distributed applications, including: consistent caching, automated replication and migration of data, persistence, access control, and fault tolerance. It does so via a carefully designed set of interfaces that supports a hierarchy of data abstractions, ranging from flat data to C++/Java objects, and that give programmers a great of control over how their data is managed. To demonstrate the effectiveness of our design, we report on our experience porting three applications to Khazana: a distributed file system, a distributed directory service, and a shared whiteboard

Elevating commodity storage with the SALSA host translation layer

To satisfy increasing storage demands in both capacity and performance, industry has turned to multiple storage technologies, including Flash SSDs and SMR disks. These devices employ a translation layer that conceals the idiosyncrasies of their mediums and enables random access. Device translation layers are, however, inherently constrained: resources on the drive are scarce, they cannot be adapted to application requirements, and lack visibility across multiple devices. As a result, performance and durability of many storage devices is severely degraded. In this paper, we present SALSA: a translation layer that executes on the host and allows unmodified applications to better utilize commodity storage. SALSA supports a wide range of single- and multi-device optimizations and, because is implemented in software, can adapt to specific workloads. We describe SALSA's design, and demonstrate its significant benefits using microbenchmarks and case studies based on three applications: MySQL, the Swift object store, and a video server.Comment: Presented at 2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS

A Multimedia Prototype for Annotation and Illustration Using the Microsoft Foundation Class Library and C++

One of the major claims of the object-oriented programming approach is that it facilitates the development of complex programs by allowing reuse of components. Most compilers for object-oriented languages are now supplied with class libraries. In addition to those provided with the compilers, there are many others in the public domain or available from commercial suppliers. Code reuse can be maximised through the exploitation of framework class libraries for creating interactive programs. A framework library can be viewed as providing a skeleton application that can be extended and specialised through class inheritance. The evolution of application frameworks is discussed briefly in Chapter 1 with an objective to utilise one of them to develop a prototype multimedia application for annotation and illustration. This prototype is referred to as Glasgow Graphics and Sound (GGS) in this thesis. GGS deals with externally created vector or bitmap images, graphics primitives and sound objects in any sequence. GGS is designed to provide the end-users with facilities to work on external images with free-hand curves and other graphics tools, record their voice, save everything in one disk file and animate them later, if necessary. GGS has the responsibility to store different objects without knowing in advance the sequence of object types the user will create. The implementation language, C++ does not have any built-in support for object persistence. Hence, a number of techniques and strategies for adding persistence to C++ objects are reviewed in Chapter 2. The Microsoft Foundation Class (MFC) library is selected as the application framework for developing GGS and the serialization mechanism in MFC is chosen to deal with the object persistence issues. Some of the techniques for persistence, discussed in Chapter 2, are powerful but incur unacceptable overheads for lightweight applications. On the other hand, the MFC serialization is found very useful in creating transportable stream of bytes that can be stored in a file and sent away as an e-mail attachment. Chapter 3 presents the serialization internals in MFC and uncovers some undocumented details that are believed to be valuable for other MFC users. From an application programmer's viewpoint, it is straightforward to use the MFC serialization in most cases. However, the actual implementation details are complex. A sample data structure is serialized and analysed step-by-step to explain the MFC serialization mechanism. The user-friendliness of applications comes not only from an iconic user interface but also from a uniform user interface across applications. Some common user interface elements and their importance are discussed in Chapter 4 along with the document/ view architecture in MFC that separates an application's data management code from its user interface code. The multiple document interface (MDI) in GGS is based on this document/view architecture. A case study walkthrough is presented, purely from an end-user's viewpoint, to illustrate a simple use of GGS. The main classes and their hierarchy are drafted in Chapter 4 based on a high-level decomposition of GGS. Chapter 5 presents the final class hierarchy, different drawing operations and other features involving graphics primitives. Template based type-safe collection classes are used in GGS to store pointers to objects of any type. This simplifies the interaction with the document class. Basic drawing operations such as moving, deleting and highlighting graphics primitives on the screen use an efficient raster drawing mode. The implementation of view magnification together with the standard scrolling capabilities in a window is discussed that requires some special techniques. The benefits of trapping some uncommon messages from the operating system are also discussed. Chapter 5 ends with an overview of the printing process and a description of the multi-page printing features in GGS. Chapter 6 starts with a general discussion on bitmaps and metafiles. A bitmap is a complete digital representation of a picture. Each pixel in the image corresponds to one or more bits in the bitmap. A metafile, on the other hand, stores pictorial information as a series of records that correspond directly to the graphics device interface (GDI) calls. GGS can import externally created bitmaps and metafiles and treat them like any other graphic or sound objects. All commercial illustration programs do something similar. However, the motivation for developing GGS is slightly different. GGS allows the users to construct and manipulate a fairly complex picture, adding comments as they go. The process of constructing the picture is saved, not just the final picture. Sound can be an effective form of information and interface enhancement when appropriately used. It can serve purposes other than the transmission of details or factual information

Safe Class and Data Evolution in Large and Long-Lived Java Applications

There is a growing class of applications implemented in object-oriented languages that are large and complex, that exploit object persistence, and need to run uninterrupted for long periods of time. Development and maintenance of such applications can present challenges in the following interrelated areas: consistent and scalable evolution of persistent data and code, optimal build management, and runtime changes to applications. The research presented in this thesis addresses the above issues. Since Java is becoming increasingly popular platform for implementing large and long-lived applications, it was chosen for experiments. The first part of the research was undertaken in the context of the PJama system, an orthogonally persistent platform for Java. A technology that supports persistent class and object evolution for this platform was designed, built and evaluated. This technology integrates build management, persistent class evolution, and support for several forms of eager conversion of persistent objects. Research in build management for Java has resulted in the creation of a generally applicable, compiler-independent smart recompilation technology, which can be re-used in a Java IDE, or as a standalone Java-specific utility similar to make. The technology for eager object conversion that we developed allows the developers to perform arbitrarily complex changes to persistent objects and their collections. A high level of developer's control over the conversion process was achieved in part due to introduction of a mechanism for dynamic renaming of old class versions. This mechanism was implemented using minor non-standard extensions to the Java language. However, we also demonstrate how to achieve nearly the same results without modifying the language specification. In this form, we believe, our technology can be largely re-used with practically any persistent object solution for Java. The second part of this research was undertaken using as an implementation platform the HotSpot Java Virtual Machine (JVM), which is currently Sun's main production JVM. A technology was developed that allows the engineers to redefine classes on-the-fly in the running VM. Our main focus was on the runtime evolution of server-type applications, though we also address modification of applications running in the debugger. Unlike the only other similar system for Java known to us, our technology supports redefinition of classes that have methods currently active. Several policies for handling such methods have been proposed, one of them is currently operational, another one is in the experimental stage. We also propose to re-use the runtime evolution technology for dynamic fine-grain profiling of applications

Agent-Based Modeling: The Right Mathematics for the Social Sciences?

This study provides a basic introduction to agent-based modeling (ABM) as a powerful blend of classical and constructive mathematics, with a primary focus on its applicability for social science research.� The typical goals of ABM social science researchers are discussed along with the culture-dish nature of their computer experiments. The applicability of ABM for science more generally is also considered, with special attention to physics. Finally, two distinct types of ABM applications are summarized in order to illustrate concretely the duality of ABM: Real-world systems can not only be simulated with verisimilitude using ABM; they can also be efficiently and robustly designed and constructed on the basis of ABM principles. �

Programming Persistent Memory

Beginning and experienced programmers will use this comprehensive guide to persistent memory programming. You will understand how persistent memory brings together several new software/hardware requirements, and offers great promise for better performance and faster application startup times—a huge leap forward in byte-addressable capacity compared with current DRAM offerings. This revolutionary new technology gives applications significant performance and capacity improvements over existing technologies. It requires a new way of thinking and developing, which makes this highly disruptive to the IT/computing industry. The full spectrum of industry sectors that will benefit from this technology include, but are not limited to, in-memory and traditional databases, AI, analytics, HPC, virtualization, and big data. Programming Persistent Memory describes the technology and why it is exciting the industry. It covers the operating system and hardware requirements as well as how to create development environments using emulated or real persistent memory hardware. The book explains fundamental concepts; provides an introduction to persistent memory programming APIs for C, C++, JavaScript, and other languages; discusses RMDA with persistent memory; reviews security features; and presents many examples. Source code and examples that you can run on your own systems are included. What You’ll Learn Understand what persistent memory is, what it does, and the value it brings to the industry Become familiar with the operating system and hardware requirements to use persistent memory Know the fundamentals of persistent memory programming: why it is different from current programming methods, and what developers need to keep in mind when programming for persistence Look at persistent memory application development by example using the Persistent Memory Development Kit (PMDK) Design and optimize data structures for persistent memory Study how real-world applications are modified to leverage persistent memory Utilize the tools available for persistent memory programming, application performance profiling, and debugging Who This Book Is For C, C++, Java, and Python developers, but will also be useful to software, cloud, and hardware architects across a broad spectrum of sectors, including cloud service providers, independent software vendors, high performance compute, artificial intelligence, data analytics, big data, etc

Innovating Pedagogy 2015: Open University Innovation Report 4

This series of reports explores new forms of teaching, learning and assessment for an interactive world, to guide teachers and policy makers in productive innovation. This fourth report proposes ten innovations that are already in currency but have not yet had a profound influence on education. To produce it, a group of academics at the Institute of Educational Technology in The Open University collaborated with researchers from the Center for Technology in Learning at SRI International. We proposed a long list of new educational terms, theories, and practices. We then pared these down to ten that have the potential to provoke major shifts in educational practice, particularly in post-school education. Lastly, we drew on published and unpublished writings to compile the ten sketches of new pedagogies that might transform education. These are summarised below in an approximate order of immediacy and timescale to widespread implementation