Performance Models of Data Parallel DAG Workflows for Large Scale Data Analytics

Directed Acyclic Graph (DAG) workflows are widely used for large-scale data analytics in cluster-based distributed computing systems. Building an accurate performance model for a DAG on data-parallel frameworks (e.g., MapReduce) is critical to implement autonomic self-management big data systems. An accurate performance model is challenging because the allocation of pre-emptable system resources among parallel jobs may dynamically vary during execution. This resource allocation variation during execution makes it difficult to accurately estimate the execution time. In this paper, we tackle this challenge by proposing a new cost model, called Bottleneck Oriented Estimation (BOE), to estimate the allocation of preemptable resources by identifying the bottleneck to accurately predict task execution time. For a DAG workflow, we propose a state-based approach to iteratively use the resource allocation property among stages to estimate the overall execution plan. Extensive experiments were performed to validate these cost models with HiBench and TPC-H workloads. The BOE model outperforms the state-of-the-art models by a factor of five for task execution time estimation.Peer reviewe

Recovery Model for Survivable System through Resource Reconfiguration

A survivable system is able to fulfil its mission in a timely manner, in the presence of attacks, failures, or accidents. It has been realized that it is not always possible to anticipate every type of attack or failure or accident in a system, and to predict and protect against those threats. Consequently, recovering back from any damage caused by threats becomes an important attention to be taken into account. This research proposed another recovery model to enhance system survivability. The model focuses on how to preserve the system and resume its critical service while incident occurs by reconfiguring the damaged critical service resources based on available resources without affecting the stability and functioning of the system. There are three critical requisite conditions in this recovery model: the number of pre-empted non-critical service resources, the response time of resource allocation, and the cost of reconfiguration, which are used in some scenarios to find and re-allocate the available resource for the reconfiguration. A brief specifications using Z language are also explored as a preliminary proof before the implementation .. To validate the viability of the approach, two instance cases studies of real-time system, delivery units of post office and computer system of a company, are provided in ensuring the durative running of critical service. The adoption of fault-tolerance and survivability using redundancy re-allocation in this recovery model is discussed from a new perspective. Compared to the closest work done by other researchers, it is shown that the model can solve not only single fault and can reconfigure the damage resource with minimum disruption to other services

Channel adaptive fair queueing for scheduling integrated voice and data services in multicode CDMA systems

CDMA (code division multiple access) systems are critical building blocks of future high performance wireless and mobile computing systems. While CDMA systems are very mature for voice services, their potentials in delivering high quality data services are yet to be investigated. One of the most crucial component in an advanced wideband CDMA system is the judicious allocation of bandwidth resources to both voice and high data rate services so as to maximize utilization while satisfying the respective quality of service requirements. Specifically, in a multicode CDMA system, the problem is to intelligently allocate codes to the users' requests. While previous work in the literature has addressed this problem from a capacity point of view, the fairness aspect, which is also important from the users' point of view, is largely ignored. In this paper, we propose a new code allocation approach that is channel adaptive and can guarantee fairness with respect to the users' channel conditions. Simulation results show that out approach is more effective than the proportional fair approach.published_or_final_versio

A novel optimal small cells deployment for next-generation cellular networks

Small-cell-deployments have pulled cellular operators to boost coverage and capacity in high-demand areas (for example, downtown hot spots). The location of these small cells (SCs) should be determined in order to achieve successful deployments. In this paper, we propose a new approach that optimizes small cells deployment in cellular networks to achieve three objectives: reduce the total cost of network installation, balancing the allocation of resources, i.e. placement of each SC and their transmitted power, and providing optimal coverage area with a lower amount of interference between adjacent stations. An accurate formula was obtained to determine the optimum number of SC deployment (NSC). Finally, we derive a mathematical expression to calculate the critical-handoff-point (CHP) for neighboring wireless stations

Wireless for Machine Learning

As data generation increasingly takes place on devices without a wired connection, Machine Learning over wireless networks becomes critical. Many studies have shown that traditional wireless protocols are highly inefficient or unsustainable to support Distributed Machine Learning. This is creating the need for new wireless communication methods. In this survey, we give an exhaustive review of the state of the art wireless methods that are specifically designed to support Machine Learning services. Namely, over-the-air computation and radio resource allocation optimized for Machine Learning. In the over-the-air approach, multiple devices communicate simultaneously over the same time slot and frequency band to exploit the superposition property of wireless channels for gradient averaging over-the-air. In radio resource allocation optimized for Machine Learning, Active Learning metrics allow for data evaluation to greatly optimize the assignment of radio resources. This paper gives a comprehensive introduction to these methods, reviews the most important works, and highlights crucial open problems.Comment: Corrected typo in author name. From the incorrect Maitron to the correct Mairto

A Framework for Resource Allocation in Time Critical Dynamic Environments Based on Social Welfare and Local Search and its Application to Healthcare

This thesis provides an artificial intelligence approach for the problem of resource allocation in time-critical dynamic environments. Motivated by healthcare scenarios such as mass casualty incidents, we are concerned with making effective decisions about allocating to patients the limited resources of ambulances, doctors and other medical staff members, in real-time, under changing circumstances. We cover two distinct stages: the Ambulance stage (at the location of the incident) and the Hospital stage (where the patient requires treatment). Our work addresses both determining the best allocation and supporting decision making (for medical staff to explore possible options). Our approach uses local search with social welfare functions in order to find the best allocations, making use of a centralized tracking of patients and resources. We also clarify how sensing can assist in updating the central system with new information. A key concept in our solution is that of a policy that attempts to minimize cost and maximize utility. To confirm the value of our approach, we present a series of detailed simulations of ambulance and hospital scenarios, and compare algorithms with competing principles of allocation (e.g. sickest first) and societal preferences (e.g. egalitarian allotment). In all, we offer a novel direction for resource allocation that is principled and that offers quantifiable feedback for professionals who are engaged in making resource allocation decisions

Management challenges arising from COVID-19: reorganization of a healthcare setting and insights for patiet transfer workflows

The condition for the healthcare services’ provision, in a context of healthcare reorganization without increasing resources, is identified in the correct allocation of patients. During the COVID-19 pandemic and also in the times to come, access to high-intensity care will be subject to several challenges. There is, therefore, the need to promptly recognize the degree of patients complexity. It is essential to define criteria that make possible the identification of the patient correct allocation from the first access to the facility. In general, patient allocation should follow a multidisciplinary approach based both on the framing of needs by health professionals, and on an assessment that can make improvements to the health performance of the organization. Starting from the identification of three gaps in the literature: (1) low research rate for the study of inter-departmental workflows; (2) low research rate for studying the elements of patient transfer management; (3) and low research rate to analyze the coordination between medical and non-medical professionals, this work aims to identify the critical issues related to the introduction of a new setting in a hospital in Central Italy. To achieve the objective, a qualitative research study has been carried out. The results highlighted two aspects relating to the criticalities of the new structure. These will need to be taken into account to best support patient transfer activities. Analyzing clinical, organizational, managerial, and technical problems in an integrated way will help improve the management efficiency of healthcare organizations as a whole

Reliable Multicast D2D Communication over Multiple Channels in Underlay Cellular Networks

Author's accepted manuscript© 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Multicast device-to-device (D2D) communications operating underlay with cellular networks is a spectral efficient technique for disseminating data to the nearby receivers. However, due to critical challenges such as, mitigating mutual interference and unavailability of perfect channel state information (CSI), the resource allocation to multicast groups needs significant attention. In this work, we present a framework for joint channel assignment and power allocation strategy to maximize the sum rate of the combined network. The proposed framework allows access of multiple channels to the multicast groups, thus improving the achievable rate of the individual groups. Furthermore, fairness in allocating resources to the multicast groups is also ensured by augmenting the objective with a penalty function. In addition, considering imperfect CSI, the framework guarantees to provide rate above a specified outage for all the users. The formulated problem is a mixed integer nonconvex program which requires exponential complexity to obtain the optimal solution. To tackle this, we first introduce auxiliary variables to decouple the original problem into smaller power allocation problems and a channel assignment problem. Next, with the aid of fractional programming via a quadratic transformation, we obtain an efficient power allocation solution by alternating optimization. The solution for channel assignment is obtained by convex relaxation of integer constraints. Finally, we demonstrate the merit of the proposed approach by simulations, showing a higher and a more robust network throughput. Index Terms—D2D multicast communications, resource allocation, imperfect CSI, fractional programming.acceptedVersio

Queuing analysis of dynamic resource allocation for virtual routers

International audienceThe most critical issue in network virtualization is the dynamic resource allocation of the physical substrate. There is a need to monitor the running virtual routers in order to allow an adaptive change in the resource allocation. In this paper, we focus on the router data plane virtualization and we explore this issue by presenting a new dynamic allocation approach through queuing theory. We consider the problem where multiple instances of virtual routers (VRs) that have some quality of service (QoS) requirements are sharing different physical resources. We propose a novel router architecture that offers a strong isolation between concurrent VRs and provides a dynamic allocation scheme in order to guarantee the provided QoS to each of them. Our approach aims at providing a higher isolation for the concurrent virtual routers sharing the same infrastructure. We propose a dynamic Weighted round robin (WRR) scheduler for each physical resource and an algorithm for adjusting the weight of each VR in order to reduce the delay of the packet processing and avoid the bottlenecks. We also propose an admission control mechanism that estimates the current load of the physical node and decides either to accept or reject a creation request of a new VR. Simulation results show that the proposed approach achieves good performance in terms of delay minimization inside the virtual router

Forecasting age-specific breast cancer mortality using functional data models

Accurate estimates of future age-specific incidence and mortality are critical for allocation of resources to breast cancer control programs and evaluation of screening programs. The purpose of this study is to apply functional data analysis techniques to model age-specific breast cancer mortality time trends, and forecast entire age-specific mortality function using a state-space approach. We use yearly unadjusted breast cancer mortality rates in Australia, from 1921 to 2001 in 5 year age groups (45 to 85+). We use functional data analysis techniques where mortality and incidence are modeled as curves with age as a functional covariate varying by time. Data is smoothed using nonparametric smoothing methods then decomposed (using principal components analysis) to estimate basis functions that represent the functional curve. Period effects from the fitted functions are forecast then multiplied by the basis functions, resulting in a forecast mortality curve with prediction intervals. To forecast, we adopt a state-space approach and an extension of the Pegels modeling framework for selecting among exponential smoothing methods. Overall, breast cancer mortality rates in Australia remained relatively stable from 1960 to the late 1990's but declined over the last few years. A set of K=4 basis functions minimized the mean integrated squared forecasting error (MISFE) and accounts for 99.3% of variation around the mean mortality curve. 20 year forecast suggest a continual decline at a slower rate and stabilize beyond 2010 and by age, forecasts show a decline in all age groups with the greatest decline in older women. We illustrate the utility of a new modelling and forecasting approach to model breast cancer mortality rates using a functional model of age. The methods have the potential to incorporate important covariates such as Hormone Replacement Therapy (HRT) and interventions to represent mammographic screening. This would be particularly useful for evaluating the impact of screening on mortality and incidence from breast cancer.Mortality, Breast Cancer, Forecasting, Functional Data Analysis, Exponential Smoothing