A relative efficiency is defined as Ridge Estimate in the general linear model. The relative efficiency is based on the Mean square error. In this paper, we put forward a parameter of Ridge Estimate and discussions are made on the relative efficiency between the ridge estimation and the General Ridge Estimate. Eventually, this paper proves that the estimation is better than the general ridge estimate, which is based on the MSE

Bao Guang Tian

Chao Yuan

English

ZENODO

   Abstract—A relative efficiency is defined as Ridge Estimate in the general linear model. The relative efficiency is based on the Mean square error. In this paper, we put forward a parameter of Ridge Estimate and discussions are made on the relative efficiency between the ridge estimation and the General Ridge Estimate. Eventually, this paper proves that the estimation is better than the general ridge estimate, which is based on the MSE.  Keywords—Ridge estimate, generalized ridge estimate, MSE, relative efficiency. I. INTRODUCTION HE generalized Markov-Gauss model [9] is described by  Y X             (1)  where Y is the n observation vector, X is a n p  column full rank design matrix ,  is an unknown parameter vector. ( ) 0E   ,   is the 1n observation vector. 2( )Cov    ,  is the n n  positive definite co variance matrix and the rank is ( )r m n   . 2 is unknown parameter. In the light of the least square unified theory of estimation [1], the BLU of   is  ( )Q y X     1 1' 'X X X y      1min ( ) min( ) ' ( )Q y X y X         the generalized least eigenvalue of   is the GLS .  1 1 1ˆ ( ' ) 'GLS X X X y                           (2)  and 2 1 1ˆ ( ' )Cov X X       22 1 11ˆ ( ' )piMSE tr X X          C. Yuan and B. Tian are with the Department of Mathematics, Qingdao University of Science & Technology, Shandong 266061, China, e-mail: (675593912@qq.com). where 1 2 0p      , i represents the characteristic value of 1'X X . In 1970, Hoerl and Kennard [2], [3] put forward the Ridge Estimate. In practical application, it is proved that the Ridge Estimate improves the performance of the LSE. Reference [4] raised a new relative efficiency and proved the Ridge Estimate was better than LSE. In this paper, the method is further applied to the new Ridge Estimate. II. THE DEFINITION AND LEMMA As is well known, ˆGLS is the BLUE for regression parameters. However, there is the multi-collinearity in model, the distance is relatively large between ˆGLS and  . Meanwhile, the estimate variance of ˆGLS  will become large. Therefore, an estimate C which is equal with the sum of squares of the residuals can be selected, ˆGLS is defined by solving the minimum estimated distance in the C . That is  11min{ ' [( ) ' ( ) ]}k y X y X c                   (3)  In model (3), it is based on the choice of different k that we get the sum of squares about different residual. Then we select the best estimate value by LSE. As a result;   11min{ ' [( ) ' ( ) ]}k y X y X c                  (4)  Theorem 1. In model (1), the Trenkler [4] Ridge Estimate (RE) is  0k  ,  1 1 1ˆ ˆ( ' ) 'RE r k GLSX X kI X y T          where 1 1 1 1( )k r k kT I ks s s ss         1's X X    1'k rs X X kI     Theorem 2. In model (1), the estimated value of the unknown parameter  is  The Relative Efficiency Based on the MSE in Generalized Ridge Estimate  Chao Yuan, Bao Guang Tian TWorld Academy of Science, Engineering and TechnologyInternational Journal of Mathematical and Computational Sciences Vol:10, No:9, 2016 463International Scholarly and Scientific Research & Innovation 10(9) 2016 scholar.waset.org/1307-6892/10005775International Science Index, Mathematical and Computational Sciences Vol:10, No:9, 2016 waset.org/Publication/10005775  1 1 1ˆ( ) ( ' ') 'K X X QKQ X y                    (5)  which is the generalized RE of   where  11 2' ' ( , , , )pQ X XQ diag          1 2( , , )pK diag k k k    1 2, , , pk k k are p  ridge parameters. Consider the following: Assume that 1 2( , , )pK diag k k k  , where  ( ' )i ik k   and 1 i p  , 0c  .  , '( ' ) ' , ', 'ii i iic k ck k k cc k c                        (6)  Then this paper makes some discussions on the efficiency of ridge estimator and the general ridge estimator based on mean square error. Theorem 3. The relative efficiency is defined as   being relative to   , which is  ( )( , ) 1 ( )MSEeMSE                            (7)  If   is BLUE, then ( , ) ( )e e     . According to Theorem 3, if 0 ( , ) 1e     , then  is better than   . And ( , )e    is larger, the efficiency is higher. So, let’s prove this result, which is  0 ( ( )) ( ( )) 1e k e K        if 1 2' , ( , , , ) 'pQ        ,  then 1 1' 'Q X y                                    (8)  1 1( ) ( ) ' 'K K Q X y                             (9)  Lemma 1[6] 1ˆ( ( )) ( )pi iiMSE K f k  , where   2 222( ) ( )i i ii ii ikf kk    . Proof. According to the result (8),  1 1' 'Q X y       1( ) ( )K K A           where 1( )A K      1 21 1 2 2, , , pp pdiagk k k             and 2ˆ( ( )) ( ( )) ( ( ) )MSE K trCov K E K        ( ( )) ( )trCov K trCov A   2 1 1[ ( ' ) ']tr A X X A   221 ( )pii i ik     ˆ ˆ( ( )) ( )E K E A AE A         then 2 2( ( ) ) ( )E K A I    2 221 ( )pi ii i ikk    Above all, 1ˆ( ( )) ( )pi iiMSE K f k  .  Lemma 2 [7] If  Q  and Q is orthogonal matrix, then   ( ) ( )MSE MSE    Proof. According to the condition Q  , then  ( ) ( )MSE MSE Q    ( ) ( ) '( )MSE E         ( ) '( )E Q Q Q Q      ( ) ' ' ( )E Q Q       ( ) '( )E        ( )MSE    Above all, ( ) ( )MSE MSE  .  Lemma 3. [7] 2 2 22( ) ( )kf kk       If 22k , meanwhile, the value of ( )f k is the least value, and where 0, 0k   . Proof. According to the condition,  World Academy of Science, Engineering and TechnologyInternational Journal of Mathematical and Computational Sciences Vol:10, No:9, 2016 464International Scholarly and Scientific Research & Innovation 10(9) 2016 scholar.waset.org/1307-6892/10005775International Science Index, Mathematical and Computational Sciences Vol:10, No:9, 2016 waset.org/Publication/10005775  2 2 2 2 242 ( ) 2( )( )'( ) ( )k k k kf kk           2 2 2 232 ( ) 2( )( )k k kk       2 232 ( )( )kk      Assume that   '( ) 0f k  ,22k ,  then ( )f k  has an extreme value. If  220 k   ,  then 2 232 ( )'( ) ( )kf kk   22 2232 ( )0( )k    .  If  22k ,  then '( ) 0f k  .  Above all, if 22k , meanwhile, the value of ( )f k is the least value, and where 0, 0k   . Lemma 4. [8] In the model of (1), when 0k  , then  2 2 221( ) ( )pi ii ikMSE kk       and 0 ( ( )) 1e k  .  Lemma 5. [4] In the model of (1), when  22 2120 ppk      ,  then 0( ( )) 0e K e                                (10)  and 2 2 210 212 (1 )( )p pk kek      ,    'Q  ,  p is the max vector component of module. III. THE MAIN RESULTS Lemma. In the model of (1), when   0k  , 1 2( , , )pK diag k k k    then  0 ( ( ), ( )) 1e K k    .  Proof. When 0k  , i. If 2 2ik   , 1, 2, ,i p  assume that   ' , 1, 2, ,ik k i p   , 2 2c   ,  then ' ' '( ) ,i i i ik k k k c                                 (11)  ' '( ) ,i i ik k c k c                                  (12)  Discussion is made on the condition (1), ( ( ), ( )) 0e K k     is obvious. Discussions are made on the condition (2),     ( ( )) ( ( ))MSE k MSE K  1 1( ) ( )p pi ii if k f c   1 1( ) ( )p pi ii if k f c      According to the Lemma 2,  ( ( )) ( ( ))MSE k MSE K   ,  ( ( )) ( ( )) 0MSE k MSE K      ( ( ), ( )) 1e K k      ⅱ. Assume that there is an i ,which lets 2 2ik   . If                       2 2 '1 ,i ik k k   ,  then ( ( )) ( ( ))MSE k MSE K  1 1( ) ( )p pi i ii if k f k   1 1( ) ( )if k f k     Discussions are made on the condition (1), this paper gets    the result according to the Lemma 3,which is  1 1( ) ( ) 0if k f k    World Academy of Science, Engineering and TechnologyInternational Journal of Mathematical and Computational Sciences Vol:10, No:9, 2016 465International Scholarly and Scientific Research & Innovation 10(9) 2016 scholar.waset.org/1307-6892/10005775International Science Index, Mathematical and Computational Sciences Vol:10, No:9, 2016 waset.org/Publication/10005775  ( ( )) ( ( )) 0MSE k MSE K      thus ( ( ), ( )) 0e K k      Discussions are made on the condition (2),  ( ( )) ( ( ))MSE k MSE K  1 1( ) ( )p pi i ii if k f k   1 1 12 2( ) ( ) ( ) ( )p pi ii if k f k f k f c        According to the Lemma3, when 2 2k c    , then    1 1 1( ) ( ) 0,f k f k    2 2( ) ( ) 0p pi ii if k f c      and ( ( )) ( ( )) 0MSE k MSE K      thus ( ( ), ( )) 0e K k      Above all, 0 ( ( ), ( )) 1e K k    .  Inference. In the model of (1), when 0k  , thena   0 ( ( )) ( ( )) 1e k e K     .  Proof. According to the Lemma 4 and 5, the result is obvious. IV. CONCLUSIONS Owing to the determining theorem of unbiased estimator, the estimation of LSE in linear model is used. But there is the multi-collinearity in model, which is very difficult to obtain solution of the problem. So, people like to use the Biased estimate, for example the RE. And this estimation may be not exact solution of the problem, which will take some losses. But this paper proves that the new estimation is better than the general RE, which is based on the MSE.  On account of the relative efficiency of linear mode based on mean square error, further studies can focus on these limitations and make improvements. 1. Discussions are made on the relative efficiency of the Generalized RE and the RE, which is based on the ridge parameter that is proposed in this paper. 2. Further studies can focus on the relation with the generalized relative coefficient and efficiency. 3. Try to find another better parameter estimation in the RE model. REFERENCES  [1] A.Y. Liu, S.G. Wang. A new relative efficiency of least square estimation in linear model. Applied probability and statistics, 1989, 15(2), pp.97-104. [2] Hoerl, A.E., Kennard, R.W. Ridge regression: Biased estimation for non-orthogonal problems (J). Technometrics, 1970, 12:55-88. [3] Hoerl, A.E., Kennard, R.W. Ridge regression: Application for non-orthogonal problems (J). Technometrics, 1970, 12:69-72. [4] Q. Tang. The Relative Efficiency of General Ridge Estimate (J). Journal of Chengdu University (Natural science), 1991, 10 (2): 42-451 [5] Trenkler, G. On the performance of biased estimators in the linear regression model with correlated or heteroscedastic errors (J). Journal of Econometrics 1984, 25:179-190. [6] P.H. Wang. The Relative Efficiency of Generalized Ridge Estimate (J), Journal of Chengdu University (Natural science), 2001, 20 (2). [7] P.H. Wang. The Efficiency of Generalized Ridge Estimate (J). Journal of Quan Zhou University (Natural science) 1998, 16 (2): 13-151. [8] A.Y.L. Huang, G.J. Chen. The relative efficiency of parameter estimation in linear model (J). Applied probability and statistics, 1998, 14 (2): 159-1641. [9] C. Yuan, B.G. Tian, A new relative efficiency based on the least eigenvalue in generalized linear model (J). International science index, 2016, 9 (10).  World Academy of Science, Engineering and TechnologyInternational Journal of Mathematical and Computational Sciences Vol:10, No:9, 2016 466International Scholarly and Scientific Research & Innovation 10(9) 2016 scholar.waset.org/1307-6892/10005775International Science Index, Mathematical and Computational Sciences Vol:10, No:9, 2016 waset.org/Publication/10005775

The Relative Efficiency Based on the MSE in Generalized Ridge Estimate

https://zenodo.org/record/1127398/files/10005775.pdf

The Relative Efficiency Based on the MSE in Generalized Ridge Estimate

Abstract

Similar works

Full text

Available Versions

ZENODO