Towards Modelling and Analysing Non-Functional Properties of Systems of Systems

International audienceSystems of systems (SoS) are large-scale systems composed of complex systems with difficult to predict emergent properties. One of the most significant challenges in the engineering of such systems if how to predict their Non-Functional Properties (NFP) such as performance and security, and more specifically, how to model NFP when the overall system functionality is not available. In this paper, we identify, describe and analyse challenges to modelling and analysing the performance and security NFP of SoS. We define an architectural framework to SoS NFP prediction based on the modelling of system interactions and their impacts. We adopt an Event Driven Architecture to support this modelling, as it allows for more realistic and flexible NFP simulation, which enables more accurate NFP prediction. A framework integrating the analysis of several NFP allows for exploring the impacts of changes made to accommodate issues on one NFP on other NFPs

Improving hydrologic modeling of runoff processes using data-driven models

2021 Spring.Includes bibliographical references.Accurate rainfall–runoff simulation is essential for responding to natural disasters, such as floods and droughts, and for proper water resources management in a wide variety of fields, including hydrology, agriculture, and environmental studies. A hydrologic model aims to analyze the nonlinear and complex relationship between rainfall and runoff based on empirical equations and multiple parameters. To obtain reliable results of runoff simulations, it is necessary to consider three tasks, namely, reasonably diagnosing the modeling performance, managing the uncertainties in the modeling outcome, and simulating runoff considering various conditions. Recently, with the advancement of computing systems, technology, resources, and information, data-driven models are widely used in various fields such as language translation, image classification, and time-series analysis. In addition, as spatial and temporal resolutions of observations are improved, the applicability of data-driven models, which require massive amounts of datasets, is rapidly increasing. In hydrology, rainfall–runoff simulation requires various datasets including meteorological, topographical, and soil properties with multiple time steps from sub-hourly to monthly. This research investigates whether data-driven approaches can be effectively applied for runoff analysis. In particular, this research aims to explore if data-driven models can 1) reasonably evaluate hydrologic models, 2) improve the modeling performance, and 3) predict hourly runoff using distributed forcing datasets. The details of these three research aspects are as follows: First, this research developed a hydrologic assessment tool using a hybrid framework, which combines two data-driven models, to evaluate the performance of a hydrologic model for runoff simulation. The National Water Model, which is a fully distributed hydrologic model, was used as the physical-based model. The developed assessment tool aims to provide easy-to-understand performance ratings for the simulated hydrograph components, namely, the rising and recession limbs, as well as for the entire hydrograph, against observed runoff data. In this research, four performance ratings were used. This is the first research that tries to apply data-driven models for evaluating the performance of the National Water Model and the results are expected to reasonably diagnose the model's ability for runoff simulations based on a short-term time step. Second, correction of errors inherent in the predicted runoff is essential for efficient water management. Hydrologic models include various parameters that cannot be measured directly, but they can be adjusted to improve the predictive performance. However, even a calibrated model still has obvious errors in predicting runoff. In this research, a data-driven model was applied to correct errors in the predicted runoff from the National Water Model and improve its predictive performance. The proposed method uses historic errors in runoff to predict new errors as a post-processor. This research shows that data-driven models, which can build algorithms based on the relationships between datasets, have strong potential for correcting errors and improving the predictive performance of hydrologic models. Finally, to simulate rainfall-runoff accurately, it is essential to consider various factors such as precipitation, soil property, and runoff coming from upstream regions. With improvements in observation systems and resources, various types of forcing datasets, including remote-sensing based data and data-assimilation system products, are available for hydrologic analysis. In this research, various data-driven models with distributed forcing datasets were applied to perform hourly runoff predictions. The forcing datasets included different hydrologic factors such as soil moisture, precipitation, land surface temperature, and base flow, which were obtained from a data assimilation system. The predicted results were evaluated in terms of seasonal and event-based performances and compared with those of the National Water Model. The results demonstrated that data-driven models for hourly runoff forecasting are effective and useful for short-term runoff prediction and developing flood warning system during wet season

Energy-Efficient Acceleration of Asynchronous Programs.

Asynchronous or event-driven programming has become the dominant programming model in the last few years. In this model, computations are posted as events to an event queue from where they get processed asynchronously by the application. A huge fraction of computing systems built today use asynchronous programming. All the Web 2.0 JavaScript applications (e.g., Gmail, Facebook) use asynchronous programming. There are now more than two million mobile applications available between the Apple App Store and Google Play, which are all written using asynchronous programming. Distributed servers (e.g., Twitter, LinkedIn, PayPal) built using actor-based languages (e.g., Scala) and platforms such as node.js rely on asynchronous events for scalable communication. Internet-of-Things (IoT), embedded systems, sensor networks, desktop GUI applications, etc., all rely on the asynchronous programming model. Despite the ubiquity of asynchronous programs, their unique execution characteristics have been largely ignored by conventional processor architectures, which have remained heavily optimized for synchronous programs. Asynchronous programs are characterized by short events executing varied tasks. This results in a large instruction footprint with little cache locality, severely degrading cache performance. Also, event execution has few repeatable patterns causing poor branch prediction. This thesis proposes novel processor optimizations exploiting the unique execution characteristics of asynchronous programs for performance optimization and energy-efficiency. These optimizations are designed to make the underlying hardware aware of discrete events and thereafter, exploit the latent Event-Level Parallelism present in these applications. Through speculative pre-execution of future events, cache addresses and branch outcomes are recorded and later used for improving cache and branch predictor performance. A hardware instruction prefetcher specialized for asynchronous programs is also proposed as a comparative design direction.PhDComputer Science and EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/120780/1/gauravc_1.pd

A Novel Distributed Representation of News (DRNews) for Stock Market Predictions

In this study, a novel Distributed Representation of News (DRNews) model is developed and applied in deep learning-based stock market predictions. With the merit of integrating contextual information and cross-documental knowledge, the DRNews model creates news vectors that describe both the semantic information and potential linkages among news events through an attributed news network. Two stock market prediction tasks, namely the short-term stock movement prediction and stock crises early warning, are implemented in the framework of the attention-based Long Short Term-Memory (LSTM) network. It is suggested that DRNews substantially enhances the results of both tasks comparing with five baselines of news embedding models. Further, the attention mechanism suggests that short-term stock trend and stock market crises both receive influences from daily news with the former demonstrates more critical responses on the information related to the stock market {\em per se}, whilst the latter draws more concerns on the banking sector and economic policies.Comment: 25 page

Real-Time Context-Aware Microservice Architecture for Predictive Analytics and Smart Decision-Making

The impressive evolution of the Internet of Things and the great amount of data flowing through the systems provide us with an inspiring scenario for Big Data analytics and advantageous real-time context-aware predictions and smart decision-making. However, this requires a scalable system for constant streaming processing, also provided with the ability of decision-making and action taking based on the performed predictions. This paper aims at proposing a scalable architecture to provide real-time context-aware actions based on predictive streaming processing of data as an evolution of a previously provided event-driven service-oriented architecture which already permitted the context-aware detection and notification of relevant data. For this purpose, we have defined and implemented a microservice-based architecture which provides real-time context-aware actions based on predictive streaming processing of data. As a result, our architecture has been enhanced twofold: on the one hand, the architecture has been supplied with reliable predictions through the use of predictive analytics and complex event processing techniques, which permit the notification of relevant context-aware information ahead of time. On the other, it has been refactored towards a microservice architecture pattern, highly improving its maintenance and evolution. The architecture performance has been evaluated with an air quality case study

Towards Data-Driven Autonomics in Data Centers

Continued reliance on human operators for managing data centers is a major impediment for them from ever reaching extreme dimensions. Large computer systems in general, and data centers in particular, will ultimately be managed using predictive computational and executable models obtained through data-science tools, and at that point, the intervention of humans will be limited to setting high-level goals and policies rather than performing low-level operations. Data-driven autonomics, where management and control are based on holistic predictive models that are built and updated using generated data, opens one possible path towards limiting the role of operators in data centers. In this paper, we present a data-science study of a public Google dataset collected in a 12K-node cluster with the goal of building and evaluating a predictive model for node failures. We use BigQuery, the big data SQL platform from the Google Cloud suite, to process massive amounts of data and generate a rich feature set characterizing machine state over time. We describe how an ensemble classifier can be built out of many Random Forest classifiers each trained on these features, to predict if machines will fail in a future 24-hour window. Our evaluation reveals that if we limit false positive rates to 5%, we can achieve true positive rates between 27% and 88% with precision varying between 50% and 72%. We discuss the practicality of including our predictive model as the central component of a data-driven autonomic manager and operating it on-line with live data streams (rather than off-line on data logs). All of the scripts used for BigQuery and classification analyses are publicly available from the authors' website.Comment: 12 pages, 6 figure