A Novel Architecture of Software Testing based on SDN Hypervisor Technique for Big Data

There is a lack of network standard skills in present networking landscape. There is an increase in data plane granularity, data plane separation and simplifies the network devices, even networking industry has experienced a renewal with Software-Defined Networking (SDN). The device performance is improve by the linearly protocol by using SDN controller. The SDN-based software testing architecture is the basics of hypervisor approach. The application layer is initially combined with network updates, security and Quality of service. Software Defined Network (SDN) is a main feature. By using data plane communication protocol, the protocol communication is simplified. The physical switch controls the network data plane and virtual switch. The performance and efficiency are the accurate results that are achieved. Therefore, processing, storage, acquisition of big data and transmission are highly possible by SDN. The operation and design of SDN has big data impact. Hence, this method shows better results interms of accuracy, efficiency, computational time and security

Joint scheduling of tasks and network flows in big data clusters

201903 bcrcVersion of RecordPublishe

Joint scheduling of tasks and network flows in big data clusters

As an increasing number of big data processing platforms like Hadoop, Spark, and Storm appear and normally share the resources in the data center, it has been important and challenging to schedule various jobs from these platforms onto the underlying data center resources such that the overall job completion time is minimized. To solve the problem, the existing work either focus on the task-level scheduling techniques, such as Quincy and delay scheduling, or focus on the network flow scheduling techniques, such as D3 and preemptive distributed quick. These works deal with the scheduling of tasks and network flows separately and cannot achieve optimal performance. The reason is that the task scheduling without regard of the available network bandwidths may generate the task placement that causes serious network congestions and thus leads to long data transmission time. In this paper, we propose the joint scheduling technique by coordinating the task placement and the scheduling of network flows arising from these tasks. We develop a software-defined network (SDN)-based online scheduling framework which selects the task placement based on the available bandwidth on the SDN switches and at meanwhile optimally allocates the bandwidth to each data flow. Comprehensive trace-driven simulations show that the joint scheduling technique can take full use of the network bandwidth and thus reduce the job completion time by 55% on average compared with the benchmark methods.Department of Computing201903 bcrcpublished_fina