New multimedia applications require Quality of Service support, which is still not successfully implemented in current packet-switched networks implementations. This paper presents a concept of neural network predictor, suitable for prediction of short-term values of traffic volume generated by end user. The architecture is Radial Basis Function neural network, optimized with respect to a number of neurons. Testing mode of the neural network is very fast, what enables application of this tool in nodes of telecommunication network. This would help to warn a network management system on early symptoms of congestion expected in the near future and avoid the network overload

Kacprzak, Tomasz

Zaleski, Arkadiusz

Lodz University of Technology Repository

JOURNAL OF APPLIEDCOMPUTER SCIENCEVol. 18 No. 2 (2010), pp. 91-101Packet Switching Networks Traffic PredictionBased on Radial Basis Function Neural NetworkArkadiusz Zaleski, Tomasz KacprzakInstitute of Electronics, Technical University of Łódz´Wólczan´ska 211/215, 90-924 Łódz´, Polandazaleski@eranet.pl, tomasz.kacprzak@p.lodz.plAbstract. New multimedia applications require Quality of Service support,which is still not successfully implemented in current packet-switched net-works implementations. This paper presents a concept of neural networkpredictor, suitable for prediction of short-term values of traffic volume gen-erated by end user. The architecture is Radial Basis Function neural net-work, optimized with respect to a number of neurons. Testing mode of theneural network is very fast, what enables application of this tool in nodesof telecommunication network. This would help to warn a network manage-ment system on early symptoms of congestion expected in the near future andavoid the network overload.Keywords: telecommunication traffic, congestion control, neural networks,time series prediction, QoS, traffic flow.1. IntroductionDespite the enormous development of telecommunication networks, the prob-lem of network congestion is still a critical issue. New broadband access tech-nologies (ADSL2+, WiFi, UMTS-HSDPA, cable TV modems) increase networktraffic rapidly, what generates demand for speed and bandwidth in packet-switchednetworks: ATM, current and future TCP/IP Internet, frame relay, etc., [1-7]. New92 Packet Switching Networks Traffic Prediction...Internet applications and integrated multimedia services require QoS support (ser-vice and data protection), because global models like IntServ and DiffServ are stillnot implemented. Currently used techniques like TCP mechanisms are not effec-tive enough in many cases, so hardware extension is still basic solution for networkoverload prevention.The congestion is defined as a state of network elements in which they are notable to meet the performance objectives for the already established connectionsand/or for the new connection requests, [7]. In case of networks based on TCP/IPprotocols stack, the congestion control is implemented as an open-loop, dynamicwindow scheme. This is an example of so called reactive scheme, because mostof TCP mechanisms (RTT variance estimation, exponential backoff, Karn’s rule,slow-start, dynamic window sizing, fast retransmit and fast recovery) starts afterthe congestion has occurred and therefore its effectiveness is not satisfactory. Thismeans that some transferred data have been already lost or their delivery is de-layed, what is not acceptable for some applications. The only Explicit CongestionNotification mechanism for transport protocols would be very useful for networkoverload prevention, but it is not included in all TCP implementations and routers,so it has to be negotiated at connection establishment. The other solutions likeStream Control Transmission Protocol [8] or measurement based traffic shapingwith ICMP protocol support have also some serious disadvantages. SCTP was de-veloped mainly to eliminate impact of head-of-line blocking effect on transportprotocol performance, because in case of TCP protocol it causes messages trans-mission delay, what is not acceptable e.g. for signalling. Measurement based con-gestion control reduces network utilization and ICMP protocol is often used forDenial of Service attacks purpose like e.g. flooding.A better approach is to apply a control system, that should be preventive inorder to react quickly before the excessive load reaches saturation level, thus re-sulting in avoidance as well as minimization of the spread of congestion. However,this method requires reliable short-term traffic forecasting, which could serve as atool for sensing the early symptoms of approaching congestion state.The idea of traffic load forecasting is based on the assumption that similar stateat the near future moment have existed in the past. This results in a model of timeseries, where measured traffic parameter values for contiguous time intervals up tothe actual time point are used to predict the value for next time interval.In this paper we propose neural network forecasting model, suitable for predic-tion of short-term values of end user traffic descriptor parameters e.g. average ormaximum bit rate for particular flow within assumed time interval. The architec-A. Zaleski, T. Kacprzak 93Figure 1. Intelligent agent-based system for congestion controlture is Radial Basis Function neural network, optimized with respect to a numberof neurons. Testing mode (prediction process) of the neural network is very fast,which enables application of this tool in a broadband telecommunication network.2. Agent-based system for congestion avoidanceDespite the vast research efforts and the large number of different congestionrecovery schemes proposed, there is so far no universally acceptable one. In thispaper we propose an intelligent agent-based structure for traffic prediction, whichcould help to avoid congestion in packet-switched telecommunication networks.The agent could be implemented both in network nodes as well as a separate ap-plication server connected via SNMP with a group of served network elements(hosts, routers, switches, etc.) and its activity is limited to nodes belonging to thespecified cluster. The agent structure is composed of three blocks: Traffic statisticsmodule, Control module and Neural network traffic predictor, Fig. 1.The module of traffic statistics monitors on-line flows of traffic passing theobservation point (typically it is node) and gathers necessary information of their94 Packet Switching Networks Traffic Prediction...long-term behaviour in the past. This information is next prepared in the form ofinput data for neural network traffic predictor, so the quality and proper selectionof statistics for particular traffic parameter have direct impact on speed of trainingprocess and prediction precision. The neural network is a time series predictorusing lag points from a sequence of real-time values in the past. The output ofthis block is the predicted value of specified traffic parameter, expected for nexttime interval. Based on this value and additional information the control modulecan prepare a decision, which influences traffic flows in the particular router. Thisdecision can result in, for example, changing routing procedure, controlling slidingwindow or retransmission time out.3. Neural network for time series predictionThe time series is a set of ordered data points V(t) spaced typically with uni-form time intervals, where t = 0, 1, 2, 3, ... symbolizes elapsed time. In analyzedcase mentioned data points are represented by selected traffic parameter measure-ments values. The goal of time series analysis is to detect a nature of phenomenon,which is described by a sequence of measurement data taken from the past, to pre-dict future values of this time series. Both tasks require identification and formaldescription for time series components, so that fixed pattern could be applied againfor any other data within the same model.There are a lot of time series analysis methods like spectral analysis, exponen-tial smoothing or autoregressive integrated moving average (ARIMA), [9]. Unfor-tunately, mentioned techniques can’t be used for packet traffic parameters predic-tion, due to their bursty and fast time-varying characteristics and huge amount offlows processed in core network routers. Traditional techniques provide optimalsolutions only in case of stable states, because of real time calculations require-ment and the prediction system should be able to process a lot of data in real time.Lately, more and more popular are methods based on statistics and artificialintelligence like e.g. genetic algorithms or artificial neural networks, which arealready used for many other applications like function approximation, modellingand pattern recognition, [10-12]. Using neural network for time series predictionpurpose instead of traditional techniques, could reduce errors which descend frommodelling, approximation and unpredictable changes in the monitored system. Ad-ditionally, they are able to transmit relationships discovered during training processto new cases, which were not used during the training stage (generalization). Neu-A. Zaleski, T. Kacprzak 95Figure 2. Efficiency comparison for selected networks and training algorithmsral network scheme trained by sufficiently long time series taken from the historyshould be a good solution for prediction of one point ahead and it was also pro-posed in [12], where multilayer perceptron has been applied to predict 24-hourload shape of Polish power system. The objectives of our research are to confirm ifneural network approach is a good solution for packet traffic parameters predictionand to determine the optimal neural network model for this purpose.This task requires making decision on neural network type (architecture), train-ing algorithm and training data set. Efficiency research for different solutions hasto return the optimal architecture, with number of training epochs, input data vec-tor size and number of hidden layers neurons reduced to minimum. It is effectiveon number of floating-point operations (processor capacity) demand, what couldbe important in case of large systems handling, when traffic prediction for manyusers/flows has to be performed within limited time period.For time series prediction task three types of neural network are recommended:feedforward neural networks e.g. multilayer perceptron, recurrent networks e.g.Elman network and Radial Basis Function neural network [12]. These three kindsof architectures had been extensively tested for the same training data set and theircomplexity had been compared as well as number of training epochs required toobtain an assumed mean square error value. The efficiency comparison result forexample time series prediction task was presented in Fig. 2, where the number oftraining epochs required to reduce training error to the assumed value was calcu-lated.The Radial Basis Function (RBF) neural network of the architecture given inFig. 3 has been chosen as the best solution from these three ones, due to goodgeneralization and quick training process. The RBF network used has three layers.96 Packet Switching Networks Traffic Prediction...Figure 3. Architecture of Radial Basis Function neural network under testAt the input layer 7 neurons act as distributing points. At the hidden layer 3 neuronsuse the Gaussian-like response function, while at the output layer one neuron actsas a linear summer of hidden layer weighted outputs.The radial transfer function is of the form:φ(‖x¯ − c¯‖) = exp(− ‖x¯ − c¯‖22α2)(1)where: x¯ is the input vector, c¯ is the centre (vector) and parameter α (scalar) is aspread constant of RBF function.There is no general recommendation concerning the input vector size for timeseries prediction, therefore it has to be done by trial-and-error method - in analyzedcase it has to be correlated with a typical time-varying character of the telecom-munication traffic shape. Based on extended experiments the following structureof input vector x¯ = [x1, x2, x3, x4, x5, x6, x7] was established:1. first five coordinates: x1, x2, x3, x4, x5 correspond to five values of specifiedtraffic parameter, taken from five time intervals back from the actual time point t:V(t − 5),V(t − 4),V(t − 3),V(t − 2),V(t − 1), Fig. 4,2. the x6 coordinate is the average value V¯ of traffic parameter calculatedthrough five time intervals back from the actual time point t,A. Zaleski, T. Kacprzak 97Figure 4. Diagram of time series of telecommunication traffic used for trainingthe Radial Basis Function neural network predictor. Time point (t) represents themoment, when the predicted value for next interval V(t + 1) is calculatedFigure 5. The learning pattern of 5 traffic flows of user activity3. the x7 coordinate is the variance σ2 of traffic parameter calculated throughfive time intervals back from the actual time point t.For neural network training example the end user traffic characteristic (averagebit rate) has been probed at one minute time interval. This results in a trainingseries as shown in Fig. 5, where long lasting idle intervals have been removed, sothe traffic characteristic represents only busy activity of the user.98 Packet Switching Networks Traffic Prediction...Figure 6. The input data array for (n) point values prediction. Time point (t) repre-sents the moment, when the prediction of estimated values for (n) points is started.The training series is used to generate a large number of individual samples,organized in a individual input vector x¯, Fig. 6. Each vector consists of five valuesof traffic parameter, e.g. average bit rate, observed for five time intervals back fromthe actual time instant t and two additional values, i.e. average and variance of thissequence. After presentation of the input vector x¯1 the predicted value V ′(t + 1) ofthe time series for the t+1 time interval is calculated by the neural network and theresult is compared with the correct learning value V(t + 1), which is embedded incoordinates of the next input vector x¯2. This way the learning error is determinedand then minimized in the next epoch.4. Training methodThe radial network training process requires solution of three tasks performedin sequence: selection of radial function centres c¯n, determination of radius param-eters αn and weights calculation of the output layer linear neuron. The problem ofoutput layer neurons weights adaptation is made by the so called Green matrixapproach exposed to pseudoinversion. The weight vector has the following form:w¯ = G+d¯, where Green matrix G includes neurons responses to input data in theform of vectors, whose number is equal to the Green matrix number of rows. Thed¯ vector represents expected output values of prediction system.A. Zaleski, T. Kacprzak 99The main problem for solution in case of Radial Basis Function neural net-works is selection of radial functions parameters, especially ci centres. This wasmade using the so-called self-organizing algorithm [12]. According to this algo-rithm, the learning data set is divided into groups called Voronoi areas, each rep-resented by central point being the mean value of group elements. The centre ofeach Voronoi area is an equivalent to the radial function centre, so the number ofareas is equal to the number of radial functions being controlled by self-organizingalgorithm. Initial number of radial functions i.e. hidden layer neurons has directimpact on modelling precision, so bigger input vector size results bigger num-ber of radial functions. The learning procedure performs area centres update aftereach presentation of one learning data vector x¯ [12]. Initial centres selection isperformed randomly using uniform distributed numbers. The updating c¯i centre isrealized using the following formula:c¯i(n + 1) = c¯i(n) + η[x¯n − c¯i(n)] (2)where the learning factor η decreases its value with elapsed time and takes valuesmuch smaller than 1. The other centres are not updated and each vector x¯ is pre-sented repeatedly up to final function centres findings.In Fig. 7 the test result of the neural network proposed for time series predic-tion (traffic characteristic of the end-user) is shown for two different Radial BasisFunctions shapes used: Gaussian-like (the most popular) and thin plate spline RBF.5. ConclusionsIn this paper the possibility of using of artificial neural networks for predic-tion of TCP/IP packet network user traffic characteristics was presented. It wassuccessfully proved, that proper selection of network type (architecture), trainingalgorithm and training data set, allows to achieve satisfactory results even in caseof dynamic, rapidly changing packet traffic source. The Radial Basis Function neu-ral network with self-organizing training algorithm are especially efficient for thispurpose, however huge number of flows carried in large packet networks, e.g. Inter-net, generate performance problems for proposed model in real applications. Thenumber of floating-point operations demand (processor capacity) can be criticalissue in case of large systems handling. Additional model for pattern recognitionand classification has to be studied for training/testing data set reduction - artificialneural network could be used once again.100 Packet Switching Networks Traffic Prediction...Figure 7. Test results of time series prediction with using the neural network pro-posed for different RBF shapes used.References[1] Cheng, R.-G., Chang, C.-J., Lin, L.-F., A QoS-Provisioning Neural FuzzyConnection Admission Controller for Multimedia High-Speed Networks,IEEE/ACM Transactions on Networking, Vol. 7, No. 1, 111-120, February1999.[2] Davoli, F., Maryni, P., A Two-Level Stochastic Approximation for AdmissionControl and Bandwidth Allocation, IEEE Journal on Selected Areas in Com-munications , Vol. 18, No. 2, February 2000.[3] Catania, V., Ficili, G., Palazzo, S., and Panno, D., A Comparative Analy-sis of Fuzzy Versus Conventional Policing Mechanisms for ATM Networks,IEEE/ACM Transactions on Networking, Department of Statistics, Vol. 4,No. 3, June 1996.[4] Bivens, J. A., Embrechts, M. J., and Szyman´ski, B. K., Network CongestionArbitration and Source Problem Prediction Using Neural Network, Smart En-gineering System Design, Vol. 4, pp. 243–252, 2002.A. Zaleski, T. Kacprzak 101[5] Kopertowski, Z., Burakowski, W., and Ba˛k, A., Artificial neural networksapplication in telecommunication, Telecommunication review, Vol. 4 1994,pp. 191–197 (in Polish).[6] Papir, Z., Telecommunication traffic and packed networks overload, WKŁ,Warsaw, 2001 (in Polish).[7] Grzech, A., Traffic management in telecommunication networks, WPW,Wrocław, 2002 (in Polish).[8] Stevard, R., Xie, Q., Morneault, K., Sharp, C., Shwarzbauer, H., Taylor, T.,Rytina, I., Kalla, M., Shang. L., and Paxton, V., Stream Control TransmissionProtocol, RFC 2960, October 2000.[9] Box, G., Jenkins, G. M., and Reinsel, G., Time Series Analysis: Forecastingand Control, Prentice Hall, February 1994.[10] Zhao, G.-F., Tang, H., Xu, W.-B., and Zhang, Y.-H., Application of NeuralNetworks for Traffic Forecasting in Telecom Networks, Proc. of Third Int. Con-ference on Machine Learning and Cybernetics, Vol. 4, pp. 2607-2611, Shang-hai, 26–29 August 2004.[11] Zaleski, A. and Kacprzak, T., Radial based neural network for traffic predic-tion in telecommunication networks, Proc. of 16th Polish Teletraffic Sympo-sium, pp. 153-156, Łódz´, 24–25 September 2009.[12] Osowski, S., Artificial neural networks for information processing, OWPW,Warsaw, 2000 (in Polish).

Packet switching networks traffic prediction based on radial basis function neural network

Abstract

Similar works

Full text

Available Versions

Lodz University of Technology Repository