Intelligent Resource Scheduling at Scale: a Machine Learning Perspective

Resource scheduling in a computing system addresses the problem of packing tasks with multi-dimensional resource requirements and non-functional constraints. The exhibited heterogeneity of workload and server characteristics in Cloud-scale or Internet-scale systems is adding further complexity and new challenges to the problem. Compared with,,,, existing solutions based on ad-hoc heuristics, Machine Learning (ML) has the potential to improve further the efficiency of resource management in large-scale systems. In this paper we,,,, will describe and discuss how ML could be used to understand automatically both workloads and environments, and to help to cope with scheduling-related challenges such as consolidating co-located workloads, handling resource requests, guaranteeing application's QoSs, and mitigating tailed stragglers. We will introduce a generalized ML-based solution to large-scale resource scheduling and demonstrate its effectiveness through a case study that deals with performance-centric node classification and straggler mitigation. We believe that an MLbased method will help to achieve architectural optimization and efficiency improvement

Predicting Frac Hits Using Artificial Intelligence; Application in Marcellus Shale

This work introduces a process of using AI neural networks, for analyzing complex datasets, in order to achieve a higher prediction accuracy in regards to frac hits, at the individual stage level, in the Marcellus Shale when compared to traditional linear methods.;We examined 63 producing wells (parent) along with 79 completed wells (child) to determine the best predictors for accurate frac hit predictions. Our dataset consists of 959 records with 77 predictors and a single binary output (YES or NO) for a frac hit occurrence. Linear methods make analyzing these 77 predictors, along with their interactions, difficult. Neural networks, specifically backpropagation learning algorithm that was used, integrated with a fuzzy pattern recognition algorithm, allow end users to analyze a seemingly endless number of predictors at one time in order to produce a model with increased prediction accuracy over linear approaches. The four techniques discussed include accepting the null hypothesis, a method we refer to as the industry standard, a modified version of the industry standard, and backpropagation algorithms.;In this work we observed a 92.9% prediction accuracy when using a backpropagation neural network. Traditional approaches for the same dataset yield overall accuracies of 73.0%, 64.8%, and 82.8% for the three approaches that are discussed, respectively. Increased prediction accuracy is important because this allows the operator to make proactive data driven decisions for changes in completion design, well spacing, shutting in the parent well prior to the offset frac, or simply doing nothing. These decisions are better justified with increased prediction accuracy, potentially saving the operator valuable time and money

Five new real-time detections of Fast Radio Bursts with UTMOST

We detail a new fast radio burst (FRB) survey with the Molonglo Radio Telescope, in which six FRBs were detected between June 2017 and December 2018. By using a real-time FRB detection system, we captured raw voltages for five of the six events, which allowed for coherent dedispersion and very high time resolution (10.24 $\mu$s) studies of the bursts. Five of the FRBs show temporal broadening consistent with interstellar and/or intergalactic scattering, with scattering timescales ranging from 0.16 to 29.1 ms. One burst, FRB181017, shows remarkable temporal structure, with 3 peaks each separated by 1 ms. We searched for phase-coherence between the leading and trailing peaks and found none, ruling out lensing scenarios. Based on this survey, we calculate an all-sky rate at 843 MHz of $98^{+59}_{-39}$ events sky$^{-1}$ day$^{-1}$ to a fluence limit of 8 Jy-ms: a factor of 7 below the rates estimated from the Parkes and ASKAP telescopes at 1.4 GHz assuming the ASKAP-derived spectral index $\alpha=-1.6$ ($F_{\nu}\propto\nu^{\alpha}$). Our results suggest that FRB spectra may turn over below 1 GHz. Optical, radio and X-ray followup has been made for most of the reported bursts, with no associated transients found. No repeat bursts were found in the survey.Comment: 13 pages, 11 figures, submitted to MNRA

Synchrophasors: Multilevel Assessment and Data Quality Improvement for Enhanced System Reliability

. This study presents a comprehensive framework for testing and evaluation of Phasor Measurement Units (PMUs) and synchrophasor systems under normal power system operating conditions, as well as during disturbances such as faults and transients. The proposed framework suggests a performance assessment to be conducted in three steps: (a) type testing: conducted in the synchrophasor calibration laboratory according to accepted industrial standards; (b) application testing: conducted to evaluate the performance of the PMUs under faults, transients, and other disturbances in power systems; (c) end-to-end system testing: conducted to assess the risk and quantify the impact of measurement errors on the applications of interest. The suggested calibration toolset (type testing) enables performance characterization of different design alternatives in a standalone PMU (e.g., length of phasor estimation windows, filtering windows, reporting rates, etc.). In conjunction with the standard performance requirements, this work defines new metrics for PMU performance evaluations under any static and dynamic conditions that may unfold in the grid. The new metrics offer a more realistic understanding of the overall PMU performance and help users choose the appropriate device/settings for the target applications. Furthermore, the proposed probabilistic techniques quantify the PMU accuracy to various test performance thresholds specified by corresponding IEEE standards, rather than having only the pass/fail test outcome, as well as the probability of specific failures to meet the standard requirements defined in terms of the phasor, frequency, and rate of change of frequency accuracy. Application testing analysis encompasses PMU performance evaluation under faults and other prevailing conditions, and offers a realistic assessment of the PMU measurement errors in real-world field scenarios and reveals additional performance characteristics that are crucial for the overall application evaluation. End-to-end system tests quantify the impact of synchrophasor estimation errors and their propagation from the PMU towards the end-use applications and evaluate the associated risk. In this work, extensive experimental results demonstrate the advantages of the proposed framework and its applicability is verified through two synchrophasor applications, namely: Fault Location and Modal Analysis. Finally, a data-driven technique (Principal Component Pursuit) is proposed for the correction and completion of the synchrophasor data blocks, and its application and effectiveness is validated in modal analyzes

Integracija "big data" analize i inženjerskog načina razmišljanja s ciljem upravljanja i kontroliranja inteligentnog opremanja i umjetnog načina podizanja nafte i plina za odabranu bušotinu s područja Hrvatske : diplomski rad

In order to reach more complex reservoir and increase ultimate recovery, engineers are searching for new technologies. One of these is intelligent completion which provides system monitoring, fluid production or injection control, and optimization. Operator can make decisions about managing completion based on real-time data coming from the downhole sensors. In addition, machine learning is becoming more popular in the oil industry. It finds application in automatization of processes and reducing time and error in decision making process. The aim of the thesis is to couple intelligent completion with machine learning (neural network) on the real example-gas well. The goal is to see if neural network can predict optimal interval control valve sizes for specific scenarios.Inženjeri kontinuirano istražuju nove tehnologije kako bi razradili kompleksnija ležišta i povećali njihov ukupni iscrpak ležišta. Jedan od načina je i inteligentno opremanje koje pruža mogućnost daljinskog nadgledanja i kontrole cjelokupnog procesa pridobivanja ugljikovodika, te optimizaciju cijelog procesa. U takvom sustavu operator donosi odluke na temelju podataka koji dolaze u stvarnom vremenu sa senzora postavljenih u bušotinu. Nadalje, strojno učenje (engl. machine learning) postaje sve popularnije i u naftnoj industriji. Primjenjuje se u automatizaciji procesa kako bi se smanjilo vrijeme i greške prilikom donošenja odluka. Cilj ovoga rada je spojiti inteligentno opremanje sa neuronskom mrežom na stvarnom primjeru plinske bušotine. Uz to, cilj je vidjeti može li neuronska mreža predvidjeti optimalne veličine intervalnog kontrolnog ventila za različite slučajeve

Machine Learning and Data Mining Applications in Power Systems

This Special Issue was intended as a forum to advance research and apply machine-learning and data-mining methods to facilitate the development of modern electric power systems, grids and devices, and smart grids and protection devices, as well as to develop tools for more accurate and efficient power system analysis. Conventional signal processing is no longer adequate to extract all the relevant information from distorted signals through filtering, estimation, and detection to facilitate decision-making and control actions. Machine learning algorithms, optimization techniques and efficient numerical algorithms, distributed signal processing, machine learning, data-mining statistical signal detection, and estimation may help to solve contemporary challenges in modern power systems. The increased use of digital information and control technology can improve the grid’s reliability, security, and efficiency; the dynamic optimization of grid operations; demand response; the incorporation of demand-side resources and integration of energy-efficient resources; distribution automation; and the integration of smart appliances and consumer devices. Signal processing offers the tools needed to convert measurement data to information, and to transform information into actionable intelligence. This Special Issue includes fifteen articles, authored by international research teams from several countries