Creating an Agent Based Framework to Maximize Information Utility

With increased reliance on communications to conduct military operations, information centric network management becomes vital. A Defense department study of information management for net-centric operations lists the need for tools for information triage (based on relevance, priority, and quality) to counter information overload, semi-automated mechanisms for assessment of quality and relevance of information, and advances to enhance cognition and information understanding in the context of missions [30]. Maximizing information utility to match mission objectives is a complex problem that requires a comprehensive solution in information classification, in scheduling, in resource allocation, and in QoS support. Of these research areas, the resource allocation mechanism provides a framework to build the entire solution. Through an agent based mindset, the lessons of robot control architecture are applied to the network domain. The task of managing information flows is achieved with a hybrid reactive architecture. By demonstration, the reactive agent responds to the observed state of the network through the Unified Behavior Framework (UBF). As information flows relay through the network, agents in the network nodes limit resource contention to improve average utility and create a network with smarter bandwidth utilization. While this is an important result for information maximization, the agent based framework may have broader applications for managing communication networks

A Graph-Partition-Based Scheduling Policy for Heterogeneous Architectures

In order to improve system performance efficiently, a number of systems choose to equip multi-core and many-core processors (such as GPUs). Due to their discrete memory these heterogeneous architectures comprise a distributed system within a computer. A data-flow programming model is attractive in this setting for its ease of expressing concurrency. Programmers only need to define task dependencies without considering how to schedule them on the hardware. However, mapping the resulting task graph onto hardware efficiently remains a challenge. In this paper, we propose a graph-partition scheduling policy for mapping data-flow workloads to heterogeneous hardware. According to our experiments, our graph-partition-based scheduling achieves comparable performance to conventional queue-base approaches.Comment: Presented at DATE Friday Workshop on Heterogeneous Architectures and Design Methods for Embedded Image Systems (HIS 2015) (arXiv:1502.07241

Scheduling for Space Tracking and Heterogeneous Sensor Environments

This dissertation draws on the fields of heuristic and meta-heuristic algorithm development, resource allocation problems, and scheduling to address key Air Force problems. The world runs on many schedules. People depend upon them and expect these schedules to be accurate. A process is needed where schedules can be dynamically adjusted to allow tasks to be completed efficiently. For example, the Space Surveillance Network relies on a schedule to track objects in space. The schedule must use sensor resources to track as many high-priority satellites as possible to obtain orbit paths and to warn of collision paths. Any collisions that occurred between satellites and other orbiting material could be catastrophic. To address this critical problem domain, this dissertation introduces both a single objective evolutionary tasker algorithm and a multi-objective evolutionary algorithm approach. The aim of both methods is to produce space object tracking schedules to ensure that higher priority objects are appropriately assessed for potential problems. Simulations show that these evolutionary algorithm techniques effectively create schedules to assure that higher priority space objects are tracked. These algorithms have application to a range of dynamic scheduling domains including space object tracking, disaster search and rescue, and heterogeneous sensor scheduling

High-Performance and Time-Predictable Embedded Computing

Nowadays, the prevalence of computing systems in our lives is so ubiquitous that we live in a cyber-physical world dominated by computer systems, from pacemakers to cars and airplanes. These systems demand for more computational performance to process large amounts of data from multiple data sources with guaranteed processing times. Actuating outside of the required timing bounds may cause the failure of the system, being vital for systems like planes, cars, business monitoring, e-trading, etc. High-Performance and Time-Predictable Embedded Computing presents recent advances in software architecture and tools to support such complex systems, enabling the design of embedded computing devices which are able to deliver high-performance whilst guaranteeing the application required timing bounds. Technical topics discussed in the book include: Parallel embedded platforms Programming models Mapping and scheduling of parallel computations Timing and schedulability analysis Runtimes and operating systems The work reflected in this book was done in the scope of the European project P SOCRATES, funded under the FP7 framework program of the European Commission. High-performance and time-predictable embedded computing is ideal for personnel in computer/communication/embedded industries as well as academic staff and master/research students in computer science, embedded systems, cyber-physical systems and internet-of-things.info:eu-repo/semantics/publishedVersio

High Performance Embedded Computing

Nowadays, the prevalence of computing systems in our lives is so ubiquitous that we live in a cyber-physical world dominated by computer systems, from pacemakers to cars and airplanes. These systems demand for more computational performance to process large amounts of data from multiple data sources with guaranteed processing times. Actuating outside of the required timing bounds may cause the failure of the system, being vital for systems like planes, cars, business monitoring, e-trading, etc. High-Performance and Time-Predictable Embedded Computing presents recent advances in software architecture and tools to support such complex systems, enabling the design of embedded computing devices which are able to deliver high-performance whilst guaranteeing the application required timing bounds. Technical topics discussed in the book include: Parallel embedded platforms Programming models Mapping and scheduling of parallel computations Timing and schedulability analysis Runtimes and operating systemsThe work reflected in this book was done in the scope of the European project P SOCRATES, funded under the FP7 framework program of the European Commission. High-performance and time-predictable embedded computing is ideal for personnel in computer/communication/embedded industries as well as academic staff and master/research students in computer science, embedded systems, cyber-physical systems and internet-of-things

Dynamics analysis and integrated design of real-time control systems

Real-time control systems are widely deployed in many applications. Theory and practice for the design and deployment of real-time control systems have evolved significantly. From the design perspective, control strategy development has been the focus of the research in the control community. In order to develop good control strategies, process modelling and analysis have been investigated for decades, and stability analysis and model-based control have been heavily studied in the literature. From the implementation perspective, real-time control systems require timeliness and predictable timing behaviour in addition to logical correctness, and a real-time control system may behave very differently with different software implementations of the control strategies on a digital controller, which typically has limited computing resources. Most current research activities on software implementations concentrate on various scheduling methodologies to ensure the schedulability of multiple control tasks in constrained environments. Recently, more and more real-time control systems are implemented over data networks, leading to increasing interest worldwide in the design and implementation of networked control systems (NCS). Major research activities in NCS include control-oriented and scheduling-oriented investigations. In spite of significant progress in the research and development of real-time control systems, major difficulties exist in the state of the art. A key issue is the lack of integrated design for control development and its software implementation. For control design, the model-based control technique, the current focus of control research, does not work when a good process model is not available or is too complicated for control design. For control implementation on digital controllers running multiple tasks, the system schedulability is essential but is not enough; the ultimate objective of satisfactory quality-of-control (QoC) performance has not been addressed directly. For networked control, the majority of the control-oriented investigations are based on two unrealistic assumptions about the network induced delay. The scheduling-oriented research focuses on schedulability and does not directly link to the overall QoC of the system. General solutions with direct QoC consideration from the network perspective to the challenging problems of network delay and packet dropout in NCS have not been found in the literature. This thesis addresses the design and implementation of real-time control systems with regard to dynamics analysis and integrated design. Three related areas have been investigated, namely control development for controllers, control implementation and scheduling on controllers, and real-time control in networked environments. Seven research problems are identified from these areas for investigation in this thesis, and accordingly seven major contributions have been claimed. Timing behaviour, quality of control, and integrated design for real-time control systems are highlighted throughout this thesis. In control design, a model-free control technique, pattern predictive control, is developed for complex reactive distillation processes. Alleviating the requirement of accurate process models, the developed control technique integrates pattern recognition, fuzzy logic, non-linear transformation, and predictive control into a unified framework to solve complex problems. Characterising the QoC indirectly with control latency and jitter, scheduling strategies for multiple control tasks are proposed to minimise the latency and/or jitter. Also, a hierarchical, QoC driven, and event-triggering feedback scheduling architecture is developed with plug-ins of either the earliest-deadline-first or fixed priority scheduling. Linking to the QoC directly, the architecture minimises the use of computing resources without sacrifice of the system QoC. It considers the control requirements, but does not rely on the control design. For real-time NCS, the dynamics of the network delay are analysed first, and the nonuniform distribution and multi-fractal nature of the delay are revealed. These results do not support two fundamental assumptions used in existing NCS literature. Then, considering the control requirements, solutions are provided to the challenging NCS problems from the network perspective. To compensate for the network delay, a real-time queuing protocol is developed to smooth out the time-varying delay and thus to achieve more predictable behaviour of packet transmissions. For control packet dropout, simple yet effective compensators are proposed. Finally, combining the queuing protocol, the packet loss compensation, the configuration of the worst-case communication delay, and the control design, an integrated design framework is developed for real-time NCS. With this framework, the network delay is limited to within a single control period, leading to simplified system analysis and improved QoC

Methodology for malleable applications on distributed memory systems

A la portada logo BSC(English) The dominant programming approach for scientific and industrial computing on clusters is MPI+X. While there are a variety of approaches within the node, denoted by the ``X'', Message Passing interface (MPI) is the standard for programming multiple nodes with distributed memory. This thesis argues that the OmpSs-2 tasking model can be extended beyond the node to naturally support distributed memory, with three benefits: First, at small to medium scale the tasking model is a simpler and more productive alternative to MPI. It eliminates the need to distribute the data explicitly and convert all dependencies into explicit message passing. It also avoids the complexity of hybrid programming using MPI+X. Second, the ability to offload parts of the computation among the nodes enables the runtime to automatically balance the loads in a full-scale MPI+X program. This approach does not require a cost model, and it is able to transparently balance the computational loads across the whole program, on all its nodes. Third, because the runtime handles all low-level aspects of data distribution and communication, it can change the resource allocation dynamically, in a way that is transparent to the application. This thesis describes the design, development and evaluation of OmpSs-2@Cluster, a programming model and runtime system that extends the OmpSs-2 model to allow a virtually unmodified OmpSs-2 program to run across multiple distributed memory nodes. For well-balanced applications it provides similar performance to MPI+OpenMP on up to 16 nodes, and it improves performance by up to 2x for irregular and unbalanced applications like Cholesky factorization. This work also extended OmpSs-2@Cluster for interoperability with MPI and Barcelona Supercomputing Center (BSC)'s state-of-the-art Dynamic Load Balance (DLB) library in order to dynamically balance MPI+OmpSs-2 applications by transparently offloading tasks among nodes. This approach reduces the execution time of a microscale solid mechanics application by 46% on 64 nodes and on a synthetic benchmark, it is within 10% of perfect load balancing on up to 8 nodes. Finally, the runtime was extended to transparently support malleability for pure OmpSs-2@Cluster programs and interoperate with the Resources Management System (RMS). The only change to the application is to explicitly call an API function to control the addition or removal of nodes. In this regard we additionally provide the runtime with the ability to semi-transparently save and recover part of the application status to perform checkpoint and restart. Such a feature hides the complexity of data redistribution and parallel IO from the user while allowing the program to recover and continue previous executions. Our work is a starting point for future research on fault tolerance. In summary, OmpSs-2@Cluster expands the OmpSs-2 programming model to encompass distributed memory clusters. It allows an existing OmpSs-2 program, with few if any changes, to run across multiple nodes. OmpSs-2@Cluster supports transparent multi-node dynamic load balancing for MPI+OmpSs-2 programs, and enables semi-transparent malleability for OmpSs-2@Cluster programs. The runtime system has a high level of stability and performance, and it opens several avenues for future work.(Español) El modelo de programación dominante para clusters tanto en ciencia como industria es actualmente MPI+X. A pesar de que hay alguna variedad de alternativas para programar dentro de un nodo (indicado por la "X"), el estandar para programar múltiples nodos con memoria distribuida sigue siendo Message Passing Interface (MPI). Esta tesis propone la extensión del modelo de programación basado en tareas OmpSs-2 para su funcionamiento en sistemas de memoria distribuida, destacando 3 beneficios principales: En primer lugar; a pequeña y mediana escala, un modelo basado en tareas es más simple y productivo que MPI y elimina la necesidad de distribuir los datos explícitamente y convertir todas las dependencias en mensajes. Además, evita la complejidad de la programacion híbrida MPI+X. En segundo lugar; la capacidad de enviar partes del cálculo entre los nodos permite a la librería balancear la carga de trabajo en programas MPI+X a gran escala. Este enfoque no necesita un modelo de coste y permite equilibrar cargas transversalmente en todo el programa y todos los nodos. En tercer lugar; teniendo en cuenta que es la librería quien maneja todos los aspectos relacionados con distribución y transferencia de datos, es posible la modificación dinámica y transparente de los recursos que utiliza la aplicación. Esta tesis describe el diseño, desarrollo y evaluación de OmpSs-2@Cluster; un modelo de programación y librería que extiende OmpSs-2 permitiendo la ejecución de programas OmpSs-2 existentes en múltiples nodos sin prácticamente necesidad de modificarlos. Para aplicaciones balanceadas, este modelo proporciona un rendimiento similar a MPI+OpenMP hasta 16 nodos y duplica el rendimiento en aplicaciones irregulares o desbalanceadas como la factorización de Cholesky. Este trabajo incluye la extensión de OmpSs-2@Cluster para interactuar con MPI y la librería de balanceo de carga Dynamic Load Balancing (DLB) desarrollada en el Barcelona Supercomputing Center (BSC). De este modo es posible equilibrar aplicaciones MPI+OmpSs-2 mediante la transferencia transparente de tareas entre nodos. Este enfoque reduce el tiempo de ejecución de una aplicación de mecánica de sólidos a micro-escala en un 46% en 64 nodos; en algunos experimentos hasta 8 nodos se pudo equilibrar perfectamente la carga con una diferencia inferior al 10% del equilibrio perfecto. Finalmente, se implementó otra extensión de la librería para realizar operaciones de maleabilidad en programas OmpSs-2@Cluster e interactuar con el Sistema de Manejo de Recursos (RMS). El único cambio requerido en la aplicación es la llamada explicita a una función de la interfaz que controla la adición o eliminación de nodos. Además, se agregó la funcionalidad de guardar y recuperar parte del estado de la aplicación de forma semitransparente con el objetivo de realizar operaciones de salva-reinicio. Dicha funcionalidad oculta al usuario la complejidad de la redistribución de datos y las operaciones de lectura-escritura en paralelo, mientras permite al programa recuperar y continuar ejecuciones previas. Este es un punto de partida para futuras investigaciones en tolerancia a fallos. En resumen, OmpSs-2@Cluster amplía el modelo de programación de OmpSs-2 para abarcar sistemas de memoria distribuida. El modelo permite la ejecución de programas OmpSs-2 en múltiples nodos prácticamente sin necesidad de modificarlos. OmpSs-2@Cluster permite además el balanceo dinámico de carga en aplicaciones híbridas MPI+OmpSs-2 ejecutadas en varios nodos y es capaz de realizar maleabilidad semi-transparente en programas OmpSs-2@Cluster puros. La librería tiene un niveles de rendimiento y estabilidad altos y abre varios caminos para trabajos futuro.Arquitectura de computador