The Case for a Factored Operating System (fos)

The next decade will afford us computer chips with 1,000 - 10,000 cores on a single piece of silicon. Contemporary operating systems have been designed to operate on a single core or small number of cores and hence are not well suited to manage and provide operating system services at such large scale. Managing 10,000 cores is so fundamentally different from managing two cores that the traditional evolutionary approach of operating system optimization will cease to work. The fundamental design of operating systems and operating system data structures must be rethought. This work begins by documenting the scalability problems of contemporary operating systems. These studies are used to motivate the design of a factored operating system (fos). fos is a new operating system targeting 1000+ core multicore systems where space sharing replaces traditional time sharing to increase scalability. fos is built as a collection of Internet inspired services. Each operating system service is factored into a fleet of communicating servers which in aggregate implement a system service. These servers are designed much in the way that distributed Internet services are designed, but instead of providing high level Internet services, these servers provide traditional kernel services and manage traditional kernel data structures in a factored, spatially distributed manner. The servers are bound to distinct processing cores and by doing so do not fight with end user applications for implicit resources such as TLBs and caches. Also, spatial distribution of these OS services facilitates locality as many operations only need to communicate with the nearest server for a given service

Network Optimizations for Distributed Storage Networks

Distributed file systems enable the reliable storage of exabytes of information on thousands of servers distributed throughout a network. These systems achieve reliability and performance by storing three or more copies of data in different locations across the network. The management of these copies of data is commonly handled by intermediate servers that track and coordinate the placement of data in the network. This introduces potential network bottlenecks, as multiple transfers to fast storage nodes can saturate the network links connecting intermediate servers to the storage. The advent of open Network Operating Systems presents an opportunity to alleviate this bottleneck, as it is now possible to treat network elements as intermediate nodes in this distributed file system and have them perform the task of replicating data across storage nodes. In this thesis, we propose a new design paradigm for distributed file systems, driven by a new fundamental component of the system which runs on network elements such as switches or routers. We describe the component’s architecture and how it can be integrated into existing distributed file systems to increase their performance. To measure this performance increase over current approaches, we emulate a distributed file system by creating a block-level storage array distributed across multiple iSCSI targets presented in a network. Furthermore we emulate more complicated redundancy schemes likely to be used in distributed file systems in the future to determine what effect this approach may have on those systems and what benefits it offers. We find that this new component offers a decrease in request latency proportional to the number of storage nodes involved in the request. We also find that the benefits of this approach are limited by the ability of switch hardware to process incoming data from the request, but that these limitations can be surmounted through the proposed design paradigm

Enabling Runtime Self-Coordination of Reconfigurable Embedded Smart Cameras in Distributed Networks

Smart camera networks are real-time distributed embedded systems able to perform computer vision using multiple cameras. This new approach is a confluence of four major disciplines (computer vision, image sensors, embedded computing and sensor networks) and has been subject of intensive work in the past decades. The recent advances in computer vision and network communication, and the rapid growing in the field of high-performance computing, especially using reconfigurable devices, have enabled the design of more robust smart camera systems. Despite these advancements, the effectiveness of current networked vision systems (compared to their operating costs) is still disappointing; the main reason being the poor coordination among cameras entities at runtime and the lack of a clear formalism to dynamically capture and address the self-organization problem without relying on human intervention. In this dissertation, we investigate the use of a declarative-based modeling approach for capturing runtime self-coordination. We combine modeling approaches borrowed from logic programming, computer vision techniques, and high-performance computing for the design of an autonomous and cooperative smart camera. We propose a compact modeling approach based on Answer Set Programming for architecture synthesis of a system-on-reconfigurable-chip camera that is able to support the runtime cooperative work and collaboration with other camera nodes in a distributed network setup. Additionally, we propose a declarative approach for modeling runtime camera self-coordination for distributed object tracking in which moving targets are handed over in a distributed manner and recovered in case of node failure

Modelling and simulation of Ethernet based networked mechanical systems

Distributed control and simulation based on Ethernet networks has become increasingly common in recent years due to its superior performance and cost over other networking technology. In order to systematically design such distributed systems, which combine Ethernet network components and mechanical systems, a modular modelling and simulation approach for this class of systems is necessary. In this thesis, an innovative modular modelling and simulation approach is developed to predict the behaviour of distributed mechanical systems based on Ethernet (IEEE 802.3 protocol) networks. The main objective is to predict the overall time delay of transmitted packets and estimate the real-time performance of Ethernet/mechanical systems in different topologies for both normal and abnormal operating conditions. The approach is based on discrete finite state machine (FSM) models developed for the main types of Ethernet network components (NIC, bus, hub, and switch). The FSM models are implemented in C++ and encapsulated using Simulink S-functions. This allows the network models to be easily combined with Simulink mechanical system models that are common in control and simulation applications. The new approach is tested for a number of case studies to test its capability to predict time delays that arise from network communication. Finally, combined network/mechanical simulations are performed to illustrate the application of the new approach to distributed simulation problems. Together, these results provide a new approach for simulation of Ethernet based mechanical systems that can be used to design distributed control and simulation systems

The Role of Operating Systems in Object-Oriented Distributed Multimedia Platforms

Considerable progress has been madein# the development of technologies tosupport# distributed multimedia computing, but applicationplatforms# are needed to make such technology accessible# to application programmers. This papersets# out requirements for such platforms and discusses the role of operating systems in object-oriented# platform support. We describe an initial approach to providing such a platform which attempted to factor out all realtime aspects of multimedia support ontoa# hardware based multimedia network interface unit. This solution has been partially successful but lacks the flexibility and level of integration subsequentlydemanded# by application programmers. Our response has been to redesign the multimedia network interface unit, placing more emphasis on the provision ofdynamic# services which require real-time operating system support. The new design described in this paper exploits and extends features of a distributed object-oriented micro-kernel to provide the n..

Migration of an operating system to an object-oriented paradigm

Operating System design has moved from monolithic systems such as UNIX, where all system services are implemented in a single kernel, to microkernel designs where the majority of system services are conducted in user space. A recent trend in operating system design has been to use architectural models based upon the object-oriented paradigms. This approach promotes the modelling of system resources and resource management as an organized collection of objects in such a way that the mechanisms, policites, algorithms, and data representations of the operating system are suitably encapsulated by the objects. Much of the research in this area to date has concentrated on the uses and benefits of object-oriented operating systems m the distributed systems arena. Similarly, almost all of these systems have been designed from the ground-up. I beheve that the progression towards object-oriented operating systems is likely to involve current operating systems incorporating and assimilating object-oriented features into their existing designs in a gradual manner, rather than an overnight switch to a new technology. In this light, the purpose of my thesis is to take an existing operating system and to propose a design which would migrate the original operating systems' facilities and features to an object-oriented paradigm. This thesis also evaluates the advantages and disadvantages of such a design over the existing one. Finally, future enhancements and directions are proposed based on the new operating system design

Data-Driven Distributed Modeling, Operation, and Control of Electric Power Distribution Systems

The power distribution system is disorderly in design and implementation, chaotic in operation, large in scale, and complex in every way possible. Therefore, modeling, operating, and controlling the distribution system is incredibly challenging. It is required to find solutions to the multitude of challenges facing the distribution grid to transition towards a just and sustainable energy future for our society. The key to addressing distribution system challenges lies in unlocking the full potential of the distribution grid. The work in this dissertation is focused on finding methods to operate the distribution system in a reliable, cost-effective, and just manner. In this PhD dissertation, a new data-driven distributed ($D^3M$) framework using cellular computational networks has been developed to model power distribution systems. Its performance is validated on an IEEE test case. The results indicate a significant enhancement in accuracy and performance compared to the state-of-the-art centralized modeling approach. This dissertation also presents a new distributed and data-driven optimization method for volt-var control in power distribution systems. The framework is validated for voltage control on an IEEE test feeder. The results indicate that the system has improved performance compared to the state-of-the-art approach. The PhD dissertation also presents a design for a real-time power distribution system testbed. A new data-in-the-loop (DIL) simulation method has been developed and integrated into the testbed. The DIL method has been used to enhance the quality of the real-time simulations. The assets combined with the testbed include data, control, and hardware-in-the-loop infrastructure. The testbed is used to validate the performance of a distribution system with significant penetration of distributed energy resources

Programming the Interactions of Collective Adaptive Systems by Relying on Attribute-based Communication

Collective adaptive systems are new emerging computational systems consisting of a large number of interacting components and featuring complex behaviour. These systems are usually distributed, heterogeneous, decentralised and interdependent, and are operating in dynamic and possibly unpredictable environments. Finding ways to understand and design these systems and, most of all, to model the interactions of their components, is a difficult but important endeavour. In this article we propose a language-based approach for programming the interactions of collective-adaptive systems by relying on attribute-based communication; a paradigm that permits a group of partners to communicate by considering their run-time properties and capabilities. We introduce AbC, a foundational calculus for attribute-based communication and show how its linguistic primitives can be used to program a complex and sophisticated variant of the well-known problem of Stable Allocation in Content Delivery Networks. Also other interesting case studies, from the realm of collective-adaptive systems, are considered. We also illustrate the expressive power of attribute-based communication by showing the natural encoding of other existing communication paradigms into AbC

Network Optimizations for Distributed Storage Networks

Distributed file systems enable the reliable storage of exabytes of information on thousands of servers distributed throughout a network. These systems achieve reliability and performance by storing three or more copies of data in different locations across the network. The management of these copies of data is commonly handled by intermediate servers that track and coordinate the placement of data in the network. This introduces potential network bottlenecks, as multiple transfers to fast storage nodes can saturate the network links connecting intermediate servers to the storage. The advent of open Network Operating Systems presents an opportunity to alleviate this bottleneck, as it is now possible to treat network elements as intermediate nodes in this distributed file system and have them perform the task of replicating data across storage nodes. In this thesis, we propose a new design paradigm for distributed file systems, driven by a new fundamental component of the system which runs on network elements such as switches or routers. We describe the component’s architecture and how it can be integrated into existing distributed file systems to increase their performance. To measure this performance increase over current approaches, we emulate a distributed file system by creating a block-level storage array distributed across multiple iSCSI targets presented in a network. Furthermore we emulate more complicated redundancy schemes likely to be used in distributed file systems in the future to determine what effect this approach may have on those systems and what benefits it offers. We find that this new component offers a decrease in request latency proportional to the number of storage nodes involved in the request. We also find that the benefits of this approach are limited by the ability of switch hardware to process incoming data from the request, but that these limitations can be surmounted through the proposed design paradigm