Traffic Prediction Based on Random Connectivity in Deep Learning with Long Short-Term Memory

Traffic prediction plays an important role in evaluating the performance of telecommunication networks and attracts intense research interests. A significant number of algorithms and models have been put forward to analyse traffic data and make prediction. In the recent big data era, deep learning has been exploited to mine the profound information hidden in the data. In particular, Long Short-Term Memory (LSTM), one kind of Recurrent Neural Network (RNN) schemes, has attracted a lot of attentions due to its capability of processing the long-range dependency embedded in the sequential traffic data. However, LSTM has considerable computational cost, which can not be tolerated in tasks with stringent latency requirement. In this paper, we propose a deep learning model based on LSTM, called Random Connectivity LSTM (RCLSTM). Compared to the conventional LSTM, RCLSTM makes a notable breakthrough in the formation of neural network, which is that the neurons are connected in a stochastic manner rather than full connected. So, the RCLSTM, with certain intrinsic sparsity, have many neural connections absent (distinguished from the full connectivity) and which leads to the reduction of the parameters to be trained and the computational cost. We apply the RCLSTM to predict traffic and validate that the RCLSTM with even 35% neural connectivity still shows a satisfactory performance. When we gradually add training samples, the performance of RCLSTM becomes increasingly closer to the baseline LSTM. Moreover, for the input traffic sequences of enough length, the RCLSTM exhibits even superior prediction accuracy than the baseline LSTM.Comment: 6 pages, 9 figure

When Traffic Flow Prediction and Wireless Big Data Analytics Meet

In this article, we verify whether or not prediction performance can be improved by fitting the actual data to optimize the parameter values of a prediction model. The traffic flow prediction is an important research issue for solving the traffic congestion problem in an Intelligent Transportation System (ITS). The traffic congestion is one of the most serious problems in a city, which can be predicted in advance by analyzing traffic flow patterns. Such prediction is possible by analyzing the realtime transportation data from correlative roads and vehicles. The verification in this article is conducted by comparing the optimized and the normal time series prediction models. With the verification, we can learn that the era of big data is here and will become an important aspect for the study of traffic flow prediction to solve the congestion problem. Experimental results of a case study are provided to verify the existence of the performance improvement in the prediction, while the research challenges of this data-analytics-based prediction are presented and discussed

OFLoad: An OpenFlow-based dynamic load balancing strategy for datacenter networks

The latest tremendous growth in the Internet traffic has determined the entry into a new era of mega-datacenters, meant to deal with this explosion of data traffic. However this big data with its dynamically changing traffic patterns and flows might result in degradations of the application performance eventually affecting the network operators’ revenue. In this context there is a need for an intelligent and efficient network management system that makes the best use of the available bisection bandwidth abundance to achieve high utilization and performance. This paper proposes OFLoad, an OpenFlow-based dynamic load balancing strategy for datacenter networks that enables the efficient use of the network resources capacity. A real experimental prototype is built and the proposed solution is compared against other solutions from the literature in terms of load-balancing. The aim of OFLoad is to enable the instant configuration of the network by making the best use of the available resources at the lowest cost and complexity

OFLoad: An OpenFlow-based dynamic load balancing strategy for datacenter networks

The latest tremendous growth in the Internet traffic has determined the entry into a new era of mega-datacenters, meant to deal with this explosion of data traffic. However this big data with its dynamically changing traffic patterns and flows might result in degradations of the application performance eventually affecting the network operators’ revenue. In this context there is a need for an intelligent and efficient network management system that makes the best use of the available bisection bandwidth abundance to achieve high utilization and performance. This paper proposes OFLoad, an OpenFlow-based dynamic load balancing strategy for datacenter networks that enables the efficient use of the network resources capacity. A real experimental prototype is built and the proposed solution is compared against other solutions from the literature in terms of load-balancing. The aim of OFLoad is to enable the instant configuration of the network by making the best use of the available resources at the lowest cost and complexity

Management of fault tolerance and traffic congestion in cloud data center

In this era of ubiquitous computing, coupled with the emergence of big data and internet of things, there have been constant changes in every aspect of cloud data center communications - its network connectivity, data storage, data transfer, and architectural design. As a result of this, the amount of data transferable, and the frequency of data transfer have tremendously increased; causing device failures and traffic congestions. To cope with these changes so that performance can be sustained amidst device failures and traffic congestion, the design of fault tolerant cloud data center is important. A fault tolerant cloud data center network should be able to provide alternative paths from source to destination during failures so that there will not be abrupt fall in performance. But still with the ongoing researches in this regard, there has not been a robust cloud data center design that can boast of being suitable for alleviating the poor fault tolerance of cloud data center. In this paper, we proposed the improved versions of fat-tree interconnection hybrid designs derived from the structure called Z-fat tree; to address the issues of fault tolerance. Then, we compared these designs with single fat tree architecture with the same amount of resources for client server communication pattern such as Email application in a cloud data center. The simulation results obtained based on failed switches and links, show that our proposed hybrid designs outperformed the single fat tree design as the inter arrival time of the packets reduces

Unfair Competition Issues of Big Data in China

The sound development of the market in the data-driven economy depends on the free and fair competition of big data in the industries. Since 2015, more and more unfair competition cases concerning big data have occurred in China, such as masking advertisement, click fraud, malicious incompatibility, and gathering user’s personal data from competitors by unfair means, which can be categorized to unfair competition about illegal collection/use of competitors’ big data and about network traffic. Whether China’s current legal system of anti-unfair competition can resolve the above-mentioned disputes is concerned in this article. As the Paris Convention only regulates the basic principles of “fairness” and “honest practice” for anti-unfair competition, member states have room to develop their own legal systems according to their special economic, social and cultural conditions. In order to usher in the era of digital economy and big data and to regulate more and more unfair competition events, China amended the Anti-Unfair Competitive Law in 2017 in which a new provision for regulating the operation of e-commerce was added. This article finds that the 2017 Amendment, which is far more specific and clearer than the Paris Convention, has significantly improved China’s ability to deal with unfair competition behaviors regarding big data. However, since the patterns of unfair competition in big data are changing and “innovating” quickly and constantly, law amendments will hardly or even never catch up with the changes, so judgement of unfair competition is inherently difficult. The court cannot determine that a company constitutes unfair competition simply because its business operations have substantially reduced the performance or operating effectiveness of its competitors. When judging whether an enterprise’s competitive behavior constitutes unfair competition, no matter the court is applying one of the specific provisions or the general provision, it is essential to consider whether the enterprise has malicious and dishonest practices

Extending the VEF traces framework to model data center network workloads

Producción CientíficaData centers are a fundamental infrastructure in the Big-Data era, where applications and services demand a high amount of data and minimum response times. The interconnection network is an essential subsystem in the data center, as it must guarantee high communication bandwidth and low latency to the communication operations of applications, otherwise becoming the system bottleneck. Simulation is widely used to model the network functionality and to evaluate its performance under specific workloads. Apart from the network modeling, it is essential to characterize the end-nodes communication pattern, which will help identify bottlenecks and flaws in the network architecture. In previous works, we proposed the VEF traces framework: a set of tools to capture communication traffic of MPI-based applications and generate traffic traces used to feed network simulator tools. In this paper, we extend the VEF traces framework with new communication workloads such as deep-learning training applications and online data-intensive workloads.Ministerio de Ciencia e Innovación y Agencia Estatal de Investigación (MCIN/AEI/10.13039/501100011033) R &D Project Grant (PID2019-109001RA-I00)Publicación en abierto financiada por el Consorcio de Bibliotecas Universitarias de Castilla y León (BUCLE), con cargo al Programa Operativo 2014ES16RFOP009 FEDER 2014-2020 DE CASTILLA Y LEÓN, Actuación:20007-CL - Apoyo Consorcio BUCL

Modeling, Analysis and Application of Big Traffic Data for Intelligent Transportation Systems

University of Technology Sydney. Faculty of Engineering and Information Technology.Intelligent Transportation System (ITS), an integrated system of people, roads, and vehicles by utilizing information and communications technology, has emerged as an efficient way of improving the performance of transportation systems, enhancing travel security, and providing more choice to travelers. Recently, it has been seen that the big data era for ITS is coming due to the wide use of traffic detectors like traffic cameras and GPSs. These traffic detectors can collect various types of traffic data that significantly contribute to the development of ITS, which has the benefit of the public with convenient and safe travel. With big traffic data, data-driven methods provide powerful and theoretical support for data modeling, analysis, and applications. However, existing methods still suffer from some shortcomings. First, traffic predictors usually use black-box methods to capture the spatiotemporal correlation between traffic. As a result, it reduces the flexibility of predictors due to the time-varying spatial-temporal correlation caused by frequent variation of road conditions. Second, it is impossible to cover all urban areas with traffic detectors. Thus, data absence and data sparsity have an essential impact on the reliability of travel state monitoring in a large road network. Lastly, most big data applications are based on the centralized method for processing and analyzing data, which consume more time and computational resources, optimal decision making. These make research on big traffic data in ITS become both exciting and essential. In this thesis, a physically intuitive approach is developed for short-term traffic flow prediction that captures the time-varying spatiotemporal correlation between traffic, mainly attributed to the road network topology, travel speed, and trip distribution. Experimental results demonstrate its superior accuracy and lower computational complexity compared with its counterparts. After that, a novel methodology is presented to estimate link travel time distributions (TTDs) using end-to-end (E2E) measurements detected by the limited traffic detectors. The experimental results show that the estimated results are in excellent agreement with the empirical distributions. Lastly, a distributed scheme is proposed for taxi cruising route recommendations based on taxi demands predicted by the proposed Graph Convolutional Network (GCN) based method. Experiment and simulation are both implemented. Experimental results validate the accuracy of the proposed taxi demand predictor. Simulation results indicate that our proposed taxi recommendation scheme is better than its counterparts in the aspects of minimizing the number of vacant taxis and maximizing the global revenue of taxi drivers