The problem of the parameters estima-tion for the polynomial in the input variables regression function is formu-lated and solved. The input and output variables of the regression function are multidimensional-matrices. The pa-rameters of the regression function are assumed to be random independent multidimensional matrices with Gauss-ian distribution and known mean value and dispersion matrices. The solution to this problem is a multidimensional-matrix system of the linear algebraic equations in multidimensional-matrix unknowns – function regression pa-rameters. We have considered particu-lar case of quadratic regression func-tion, for which we have obtained for-mulas for parameters calculation. The computer simulation of the quadratic regression functions is performed for the two-dimensional matrix input and output variables

Malikova, I. G.

Mukha, V. S.

Муха, В. С.

Belarusian State University of Informatics and Radioelectronics Repository

159上合组织国家的科学研究：协同和一体化国际会议关于贝叶斯多维矩阵多项式的经验回归ON THE BAYESIAN MULTIDIMENSIONAL-MATRIX POLYNOMIAL EMPIRICAL REGRESSIONVladimir S. MukhaDoctor of Technical Sciences, Professor Irina G. MalikovaSenior LecturerBelarusian State University of Informatics and RadioelectronicsMinsk, Republic of Belarus抽象。 提出并解决了输入变量回归函数中多项式参数估计的问题。 回归函数的输入和输出变量是多维矩阵。 假设回归函数的参数是具有高斯分布，已知平均值和离散矩阵的随机独立多维矩阵。 解决该问题的方法是在多维矩阵未知数中使用线性代数方程的多维矩阵系统-函数回归参数。 我们考虑了二次回归函数的特殊情况，为此我们获得了用于参数计算的公式。 对二维矩阵输入和输出变量执行二次回归函数的计算机仿真。关键字：回归函数，参数估计，最大似然估计，贝叶斯估计，多维矩阵Abstract. The problem of the parameters estimation for the polynomial in the input variables regression function is formulated and solved. The input and output variables of the regression function are multidimensional-matrices. The parame-ters of the regression function are assumed to be random independent multidimen-sional matrices with Gaussian distribution and known mean value and dispersion matrices. The solution to this problem is a multidimensional-matrix system of the linear algebraic equations in multidimensional-matrix unknowns – function re-gression parameters. We have considered particular case of quadratic regression function, for which we have obtained formulas for parameters calculation. The computer simulation of the quadratic regression functions is performed for the two-dimensional matrix input and output variables.Keywords: regression function, parameters estimations, maximum likelihood estimations, Bayesian estimations, multidimensional matrices1. IntroductionTo date the most popular methods to estimate the parameters of the regression function are maximum likelihood method and least squares method [1, 2]. The es-timations obtained by this method have good asymptotic properties and it is the justification to their application. But usage of classical methods becomes problem-160Scientific research of the SCO countries: synergy and integrationInternational Conferenceatic in the case of a small size of the sample. In this connection Bayesian method to estimate the parameters of the regression function is more attractive. The possibility of using the samples with a small size is a significant advantage of the Bayesian ap-proach. Interest to Bayesian inference lies in the problem of optimal (dual) control of regression objects [3, 4, 5], in econometrics [6], in other areas [7, 8]. The existing investigations into Bayesian approach relate mainly to linear in the parameters and in the input variables regression functions. There are also more general results. So, in work [5] Bayesian estimations of the parameters were obtained for the regression function represented as a scalar product of the parameter vector and the vector of the basis functions. Such representation is applicable to both linear and nonlinear regression functions in the input variables. However, such an approach is bad for-malized and do not has the algorithmic generality; i. e. the mathematical expression for the vector of the basis functions is not determined and the software implementa-tion is inapplicable for any number of variables and any degree of the polynomial.In the present paper we investigate a multidimensional-matrix polynomial in the input variables regression function. In this case there are not the disadvantages pointed above. Such an effect is achieved by the new multidimensional-matrix representation of the polynomial regression function.2. Problem statementLet us consider some object with q-dimensional-matrix input variable )( jxx   , ),...,,( 21 qjjjj   , p - dimensional-matrix output variable )( i , ),...,,( 21 piiii   ,)( i , ),...,,( 21 piiii    [9, 10], and suppose that output variable η is stochastically de-pendent on input variable χ so there is conditional probability density )/( xf   . We denote )(xy    regression function η on χ and assume that dependence η on χ could be represented in the form  )(x  , where ε is p-dimensional random matrix. Let for some values nxxx ,...,, 21  of input variable χ we obtained the values nooo yyy ,2,1, ,...,,   of output variable η (observations, measurements) as follows:  zxyo )(, , n,...,1  ,                           (1)where zμ is a realization of the random matrix ε, which we will name as errors of the measurements. We will consider the Gaussian distribution of the matrix ε with zero mean value and dispersion matrix Rε.Here and below we will use the following notations for indices of multidimen-sional matrices: ,..., 21 ii  , are separate indices, ),...,,( 21)( pp iiii   is a set of p in-dices (p - multiindex), ),...,,( ),(2),(1),(),( kpppkp iiii   is a set of k p-multiindices.Let the hypothetic regression function be the polynomial of m-th degree [10]:mkpkqkkqmkkkqpkq CxxCx0),(,00),(,0 )()()( , ,...,2,1,0m          (2)161上合组织国家的科学研究：协同和一体化国际会议where ),( kqpC  and ),( pkqC  are multidimensional-matrix parameters of the regres-sion function, ),( kqpC   is  )( kqp    - multidimensional matrix:)(),()( ,),( kqp jikqpсC  , ),...,,( 21)( pp iiii  , ),...,,( )(2)(1)(),( kqqqkq jjjj   .It is symmetric relative to p-multiindices kqqq jjj )(2)(1)( ,...,,  . The matrix ),( pkqC   is transposed matrix ),( kqpC  , i. e. kqkqpHpkqkqp CC ,)( ),(),(  , kqkqpBkqppkq CC ,)( ),(),(   ,where kqkqpH ,  and kqkqpB ,   are transpose substitutions of the types ‘back’ and ‘onward’ respectively [10]. We also denote )( ),(,0 kkqpkq xC   ),0( kq  -rolled prod-uct of matrices ),( kqpC  , kq is product between k and q, kk xx 0,0  is the (0,0) - rolled degree of the matrix x [9, 10]. In these conditions the measurement ,oy   (1) has the probability density ),...,,,/( ),(),()0,(, mqpqppo CCCxyf    )))(((21exp 20),(,0,012,0mkkkqpkqpy xCyRC , n,...,1  ,  (3)where Cy is a normalizing constant, Rε-1 is (0, p) - inverse to Rε matrix [9, 10].The problem consists in finding the estimations of parameters ),( kqpC  ( ),( pkqC )   of the regression function (2) by using the given measurements ),(),...,,(),,( ,2,21,1 nonoo yxyxyx  ),(),...,,(),,( ,2,21,1 nonoo yxyxyx  .3. Bayesian multidimensional-matrix polynomial empirical regressionIn addition to the assumptions (1)–(3) we will consider the parameter ),( kqpC   of multidimensional-matrix polynomial regression (2) as a random matrix with Gaussian priori probability density))(((21exp()( 2),(,),(1),(,)(2,0),(),( kqpakqpkqpakqpkqpkqpa CCRKCf   ,  (4)mk ,...,2,1,0 , ,...2,1,0m  ,   ,where ),( kqpK   is a normalizing constant, kqkqpHpkqakqpa CC ,)( ),(,),(,   , kqkqpBkqpapkqa CC ,)( ),(,),(,    kqkqpBkqpapkqa CC ,)( ),(,),(,    is a priori mean value )( kqp   - dimensional matrix),),(),(,),(,,,)( kqkqpkqkqp HHpkqakqpa RR   , ),(),(,),(,,,)( kqkqpkqkqp BBkqpapkqa RR   , is a 162Scientific research of the SCO countries: synergy and integrationInternational Conferencepriori dispersion matrix ( )(2 kqp    - dimensional matrix), ),(1),(,1),(,,,)( kqkqpkqkqp HHpkqakqpa RR    , ),(1 ),(,1),(,,,)( kqkqpkqkqp BBkqpapkqa RR    , are ),0( kqp    -inverse to the ),(, kqpaR , ),(, pkqaR  matrices respectively. We will assume that the parameters )0,( pC , ),( qpC ,…, ),( mqpC   are independent, i. e.mkkqpamqppa CfCCf0),(),()0,( )(),...,(  .                           (5)In these assumptions on the base of measurements ),(),...,,(),,( ,2,21,1 nonoo yxyxyx   ),(),...,,(),,( ,2,21,1 nonoo yxyxyx  we will find the Bayesian estimations )0,( pC, ),( qpC, …, ),( mqpC  of the unknown parameters )0,( pC  , ),( qpC  …, i. e. the estimations minimizing the average risk:)),...,,,...,(( ),()0,(),()0,( mqpqpmqpqp CCCCWEr  ,where ),...,,,...,( ),()0,(),()0,( mqpqpmqpqp CCCCW  is loss function, E is a sym-bol of mathematical expectation.Theorem. Under conditions (1)–(3), (4), (5) relative to the multidimension-al-matrix polynomial regression and quadratic loss function the Bayesian esti-mations )0,( pC , ),( qpC ,…, ),( mqpC  of the parameters )0,( pC  , ),( qpC  ,…, ),( mqpC   satisfy the following system of linear multidimensional-matrix equations: mkkqpTkkqpqpqpaqp CVCR k0),(,),(0),(1),,(),(0 )()( ,  )()( ),,(1),,(),(01,0qpaqpaqpyxp CRSR    , m,...,1,0  , (6)where yxS  and xxkS   are defined by expressionsnyx xyS10,0 )( , nkxx xxS k10,0 )( ,  ,,kV  is )2( qkqp   -dimensional matrix,)( 10,0,   xxk kSRV  ,1R  is (0, p)-inverse to the R  matrix, ,,kTkV   is transposed in accordance with substitution Tk,λ matrix Vk,λ, and163上合组织国家的科学研究：协同和一体化国际会议),(),()()(),()(),()(,,,,,,,qkqppkqpqpkvtjitjviT .  .We do not provide a proof of the theorem. 4. Bayesian multidimensional-matrix quadratic empirical regressionAssumption m = 2 in the expression (2) gives us quadratic regression function:)()( 2)2,(2,0)1,(,0)0,( xCxCCy qppqppqp   .                     (7)In works [11, 12] one can find the algorithm of calculation of the ML-estima-tions of the parameters )0,( qpC , )1,( qpC , )2,( qpC   for the regression function (7). Here we obtain the Bayesian estimations of these parameters. The system of equations (6) for these parameters contains three equations:  )()()()( )2,(0,2)2,(0)1,(0,1)1,(0)0,(0,0)0,(0)0,(1)0,,()0,(0 0,20,10,0qpTqpqpTqpqpTqpqpqpaqp CVCVCVCR  )()( )0,,(1)0,,()0,(01,00 qpaqpaqpyxp CRSR   ,    )()()()( )2,(1,2)2,(0)1,(1,1)1,(0)0,(1,0)0,(0)1,(1)1,,()1,(0 1,21,11,0qpTqpqpTqpqpTqpqpqpaqp CVCVCVCR  )()( )1,,(1)1,,()1,(01,01 qpaqpaqpyxp CRSR   ,   )()()()( )2,(2,2)2,(0)1,(2,1)1,(0)0,(2,0)0,(0)2,(1)2,,()2,(0 2,22,12,0qpTqpqpTqpqpTqpqpqpaqp CVCVCVCR  )()( )2,,(1)2,,()2,(01,02 qpaqpaqpyxp CRSR   .  Collecting similar terms we obtain the following system of equations:  )()())(( )2,(0,2)2,(0)1,(0,1),(0)0,(0,01)0,,(,0 0,20,10,0qpTqpqpTqpqpTqpap CVCVCVR  )()( )0,,(1)0,,(,01,00 qpaqpapyxp CRSR   ,    )())(()( )2,(1,2)2,(0)1,(1,11)1,,(),(0)0,(1,0,0 1,21,11,0qpTqpqpTqpaqpqpTp CVCVRCV  )()( )1,,(1)1,,(),(01,01 qpaqpaqpyxp CRSR   ,   ))(()()( )2,(2,21)2,,()2,(0)1,(2,1),(0)0,(2,0,0 2,22,12,0qpTqpaqpqpTqpqpTp CVRCVCV  )()( )2,,(1)2,,()2,(01,02 qpaqpaqpyxp CRSR   .  164Scientific research of the SCO countries: synergy and integrationInternational ConferenceWith notations)( 0,00,01)0,,()0,(Tqpaqp VRR  , )( 1,11,11)1,,()1,(Tqpaqp VRR  , )( 2,22,21)2,,()2,(Tqpaqp VRR  ,  )()( )0,,(1)0,,(,01,0)( 0 qpaqpapyxpp CRSRB  , )()( )1,,(1)1,,(),(01,0)( 1 qpaqpaqpyxpqp CRSRB  , )()( )2,,(1)2,,()2,(01,0)2( 2 qpaqpaqpyxpqp CRSRB  ,  ,we rewrite this system in the form:.)()()(,)()()(,)()()()2()2,()2,()2,(0)1,(2,1),(0)0,(2,0,0)()2,(1,2)2,(0)1,()1,(),(0)0,(1,0,0)()2,(0,2)2,(0)1,(0,1),(0)0,()0,(,02,12,01,21,00,20,1qpqpqpqpqpTqpqpTpqpqpTqpqpqpqpqpTppqpTqpqpTqpqpqppBCRCVCVBCVCRCVBCVCVCR  This system of equations can be solved by the elimination method.5. ConclusionIn conclusion, we outline the main results of this work and note their particu-larities.1. The problem of the building of the Bayesian multidimensional-matrix polynomial regression was formulated and solved. This regression has the fol-lowing particularities compared with existing regressions: 1) the more general multidimensional-matrix polynomial regression function, when input and output variables are the multidimensional matrices, is considered; 2) a new untraditional multidimensional-matrix form of the representation of the regression function in the manner of multidimensional-matrix polynomial is used. Besides, the priory distributions of the multidimensional-matrix parameters of the regression function are supposed as Gaussian. The general solution of this problem is the system of the linear multidimensional-matrix equations relative the multidimensional-matrix parameters of the regression function. 2. On the base of the general solution the algorithm of the parameters calcula-tion of the Bayesian multidimensional-matrix quadratic empirical regression func-tions was obtained.165上合组织国家的科学研究：协同和一体化国际会议3. The computer simulation of the quadratic Bayesian empirical regressions function with two-dimensional input and output variables was performed. The simulation confirmed the correctness of the theoretical results and illustrated the important benefits of the Bayesian approach to have the algorithmic generality and to obtain the estimations on the small number of the measurements.References1. S.M. Ermakov, A.A. Zhiglyavskii, Mathematical theory of optimal experi-ment. Moscow, Publishing house “Science”, 1987. 320 p. In Russian.2. N.P. Klepikov, S.N. Sokolov, Analysis and planning of experiments by the method of maximum likelihood. Berlin, Akademie-Verlag, 1961. 127 p.3. A.A. Feldbaum, The bases of the theory of optimal automatic systems. Mos-cow, Publishing house “Science”, 1966. 552 p. In Russian.4. V.S. Mukha, E.V. Sergeev, Dual control of regression objects. Proceedings of LETI. Issue 202, 1976. Pp. 58–65. In Russian.5. V.S. Mukha, E.V. Sergeev, Bayesian estimations of the parameters of the regression objects. Proceedings of LETI. Issue 149, 1974. Pp. 26–29. In Russian.6. A. Zellner, An Introduction to Bayesian Inference in Econometrics. John Wiley and Sons, Inc. 1971. 448 p.7. Ludwig Fahrmeir, Thomas Kneib, Bayesian Smoothing and Regression for Longitudinal, Spatial and Event History Data. Oxford University Press, Oxford, 2011. 544 p.8. Jon Wakefield, Bayesian and Frequentist Regression Methods. Springer, 2013. 709 p.9. N.P. Socolov, Introduction to the theory of multidimensional matrices. Kiev, Naukova dumka, 1972. 176 p. In Russian.10. V.S. Mukha, Analysis of multidimensional data. Minsk, Technoprint, 2004. 368 p. In Russian.11. V.S. Mukha, Multidimensional-matrix polynomial regression analysis. Pa-rameters estimations. Proceedings of the National Academy of Sciences of Be-larus. Physics and Mathematics series. 2007, No. 1, pp. 45–51. In Russian.12. V.S. Mukha, The best polynomial multidimensional-matrix regression. Cy-bernetics and Systems Analysis. Vol. 43, Issue 3, 2007. Pp 427–432.

On the Bayesian multidimensional-matrix polynomial empirical regression

https://libeldoc.bsuir.by/bitstream/123456789/38988/1/Mukha_On.pdf

On the Bayesian multidimensional-matrix polynomial empirical regression

Abstract

Similar works

Full text

Available Versions

Belarusian State University of Informatics and Radioelectronics Repository