Didmanidze, Ibraim

Kakhiani, Hryhorii

English

National University of Kyiv-Mohyla Academy

y ^ K  512.7I. Didmanidze, G. KakhianiARTIFICIAL NEURAL NETWORK AND WAYS OF ITS OPTIMIZATION FOR SOME PROBLEMSTraining effectiveness o f  the Multi layer neuron networks depends on the quantity o f  neuron layers within the network, quantity o f  neurons in the hidden layer and initialization ofsynaptic connections ’weight coefficients.Keywords: neuron network, artificial neural network, synaptic connection.Neuron network’s architecture has a great signi­ficance for the effectiveness of training. The dimen­sion of the input and output layers depends on the problem for solution and the structure of the data collection. As mentioned earlier, three layer neuron network can approximate any function with the pre­liminary exactness. Besides, the exactness of the ap­proximation depends on the quantity of the neurons in the hidden layer.As mentioned earlier, the problem in the algo­rithm of error’s back propagation was that it didn’t allow for choosing preferable speed of training, which was needed for the increase of algorithm’s ra­pidity and ensure its collectability.The given equation cannot be solved with the analytic methods to apply to « ( t ) .  Therefore, [1] research suggests to apply axial flow methods in order to find adapt rapidity. However it is related with the large volume calculations. Therefore, we can suggest approximating method for finding a(t) which depends on the expansion of activation function of the neuron elements in the Taylor line. I will discuss it in details: if  we calculate the j neuron’s output value in the last layer o f the neuron networks:yj-(0 =  F  ( S j to ) ,SjO) =  S i ( i )Where y f( t )  is the output value of the hidden layer’s j neuron. To learn the value (1) of the j neuron in the t+1:+ 1 > = K  -  - Ti + a Wi == 1 ,  J i « „  - T ,  +  a -  ( S j  (2)In order to do so, I have to introduce the signifi­cations:T-, dE dEai  =  Z i y r  i ^ - ^ 7  ©9T;Which represents the figure (3) in the following way:S f t  +  1) =  5,- ( t )  -  a  ■ a ,  (4)By the (t+1) momentum the j neuron’s output value will be equal:yi ( t  +  l )  =  F ( 5 ?C t + l ) )If expand be the last figure in the Taylor line with the precision of the first two components:y j ( t  +  1) =  n o )  +  F '( 0) ■ F  { S j ( t  +  1)), (5)WhereF'{ 0)  =dFasWhen S; =  0. figure (5) considered in the figure (4) will give:y j ( t )  =  F ( 0 )  +  F r (0 ) ■ 5 j ( t )  -  a F ' f 0)«, ,  (6)because:yJ(t) = F(0)+F'(0)-5J(t).Then figure (6) can be represented the following way:y j ( t  +  1 ) =  y ,(t )F (0 }  -  c tF '(0)o , (7)In order to find the adaptive training rapidity the following should be procured:E =  +  1 ) -  tj-)Z -+ m i nthenf. = U y,®  ~ h -  “f ’O»;) ■ (-F'(o) -^) = oif we take « ( 0  from the last figure, we will compute:a ( 0  = (8)Ô * Fif —-  >  0 then for the given a(t) can be reacheddorminimum of the average square deviation. We have to find depiction for a- -  therefore, we will intro­duce the definitions:© flidMatudae I. III., Kaxianir O., 2013Дідманідзе І. Ш„ Кахіані Г. О. Нейронна мережа та шляхи її оптимізації 673E _  dE_ <hj_ dS j _9w jj dyj dSj dE dE dy i dSj f  \  , /  ■,HT; ~  ' 3 S j '  d T j ~  ~ ( y }  ~  t i^ F  ^  ^If we insert (9) and (10) in the (3) figure, following will be formulated:^  =  ( i  + 2 .  * ? ) ■ ( * - * ) ■  ^ 6 ) 0  i)Due to the principle of the layers independence we can assume thatYj =  _Vj -  % (12)with consideration o f ( l l ) in (12)  and (7) we can obtain formula to calculate approximate value of the adapt rapidity for the neuron network training.« m  = _______ _________________Where Yj ~  j" is neuron element’s error which can be calculated for various layers of the network.If have hyperbolic tangent in the role of activa­tion function, we can compute the following:y’i = F'(st) = =  C 1 -  y?lV,'(0) /"(0) 1Consequently:a ( t )  = s,y,2( i - y ; )а + 5;у?)2 ^ ( 1 -In which the error of the hidden layer’s j neuron element will be:Y jShould be mentioned, that a(t) was calculated on separate basis for each layer o f the neuron net­work in the above representations. As the experi­ment proved it, adapt rapidity o f training can be used to obtain major values o f a(t). This can lead to the de-synchronization of the training process, when weight coefficients o f the synaptic connec­tions change drastically in certain direction. As a result, the average square error’s change will acquire time dependent wave mode. It is recom­mended to limit a(t) according to the absolute value. The above figures, obtained for the adapt rapidity for training can enable us to increase training rapidity significantly and to avoid the problem of random selection of the neuron net­work’s training rapidity. This is significant priori­ty in comparison with the standard algorithm of the error’s back propagation. However, in case there is a good selection o f the training’s constant rapidity, the functioning time of the given algo­rithm will not exceed the functioning time of the gradient descending algorithm [2].References1. К вопросу об использовании искусственной нейронной се­ти для прогнозирования валютных курсов на высоко- волантильных рынках / И. Дидманидзе, 3. Мегрелишвили, 2.Г. Кахиани // Материалы I международной научно-практи­ческой интернет-конференции «Сучасні тенденції розвиткуматематики та її прикладні аспекти». -  Донецк : [б. в.], 2012. -С . 166-167.Hortz J. Introduction to the theory of neural computation / J. Hortz, A. Krogh, R. Palmer // Addison Wesley Publishing Company, 1991. -  327 p.Дідманідзе І. Ш., Кахіані Г. О.НЕЙРОННА МЕРЕЖА ТА ШЛЯХИ ї ї  ОПТИМІЗАЦІЇЕфективність багатошарових нейронних мереж залежить від кількості шарів нейронів у  межах мережі, кількості нейронів у  прихованому шарі та ініціалізації коефіцієнтів ваги синоптичних зв ’язків.Ключові слова: нейронна мережа, штучна нейронна мережа, синаптичний зв’язок.Матеріал надійшов 31.07.2013

Artificial neural network and ways of its optimization for some problems

http://ekmair.ukma.edu.ua/bitstream/handle/123456789/2863/Didmanidze_Kakhiani_Artifical_neural_network.pdf?sequence=3&isAllowed=y

Artificial neural network and ways of its optimization for some problems

Abstract

Similar works

Full text

Available Versions

National University of Kyiv-Mohyla Academy