Robust Estimation of a Linearized Nonlinear Regression Model with Heteroscedastic Errors:A Simulation Study

A simulation study is used to examine the robustness of some estimators on a linearized nonlinear regression model with heteroscedastic errors, namely the Linearized Ordinary Least Squares (LOLS), Transformed Generalized Least Squares (TGLS) , Linearized Reweighted Least Squares (LRLS) and Transformed Linearized Reweighted Least Squares (TLRLS). The latter is a modification of Reweighted Least Squares (RLS) based on Least Median of Squares (LMS). The empirical evidence shows that the first three estimators are not sufficiently robust when the percentage of outliers in the data increases. That is, they do not have a high breakdown point. On the other hand, the modified estimator (TLRLS) has a higher breakdown point than the other three estimators

Bootstrap Methods in a Class of Non-Linear Regression Models

In this paper, the performances of the bootstrap standard errors (BSE) of the Weighted MM (WMM) estimates were compared with the Monte Carlo (MCSE) and Asymptotic (ASE) standard errors. The properties of the Percentile (PB), Bias-Corrected Persentile (BCP), Bias and Accelerated (BC), Studentized Percentile (SPB) and the Symmetric (SB) bootstrap confidenceaintervals of the WMM estimates were examined and compared. The results of the study indicate that the BSE is reasonably close to the ASE and MCSE for up to 20% outliers. The BCa has attractive properties in terms of better coverage probability, equitailness and average interval length compared to the other methods

Diagnostic-robust generalized potentials for identifying high leverage points in mediation analysis

Due to the fact that mediation model involves several linear regression equations, there is concern not only when the data contain observations that are extreme in the response variable but also in the regressor space, namely the leverage points. The Diagnostic Robust Generalized Potentials (DRGP) procedure in multiple linear regression incorporated the Robust Mahanalobis Distance based on the minimum volume ellipsoid and uses Median Absolute Deviation as its cut-off points. In this paper, a slight modification to the DRGP is proposed and we call it ModDRGP. The ModDRGP is applied to the mediation model. The performance of our proposed ModDRGP is evaluated based on Monte Carlo simulation study. The simulation results suggest that ModDRGP has improved the accuracy of the identification of high leverage points when the percentage of high leverage points is medium or high. The method can also be used for the identification of high leverage points in multiple mediation models, as well

Diagnostic plot for the identification of high leverage collinearity-influential observations

High leverage collinearity influential observations are those high leverage points that change the multicollinearity pattern of a data. It is imperative to identify these points as they are responsible for misleading inferences on the fitting of a regression model. Moreover, identifying these observations may help statistics practitioners to solve the problem of multicollinearity, which is caused by high leverage points. A diagnostic plot is very useful for practitioners to quickly capture abnormalities in a data. In this paper, we propose new diagnostic plots to identify high leverage collinearity influential observations. The merit of our proposed diagnostic plots is confirmed by some well-known examples and Monte Carlo simulations

Robust nonlinear regression: case study for modeling the greenhouse gases, methane and carbon dioxide concentration in atmosphere

Four nonlinear regression models are proposed for the atmospheric carbon dioxide and methane gas concentrations data, reported by United Nation 1989. Among those considered, the Exponential with Intercept is the most preferred one to model methane data due to better convergence and lower correlation between parameters. On the other hand, the scale exponential convex model is appropriate for carbon dioxide data because besides having smaller standard errors of parameter estimates and smaller residual standard errors, it is numerically stable. Due to large range of data that goes back to history to 7000 years ago, there is a big dispersion in data set, so that it made us to apply robust nonlinear regression estimation methods to have a smoother model

The performance of mutual information for mixture of bivariate normal disatributions based on robust kernel estimation.

Mutual Information (MI) measures the degree of association between variables in nonlinear model as well as linear models. It can also be used to measure the dependency between variables in mixture distribution. The MI is estimated based on the estimated values of the joint density function and the marginal density functions of X and Y. A variety of methods for the estimation of the density function have been recommended. In this paper, we only considered the kernel method to estimate the density function. However, the classical kernel density estimator is not reliable when dealing with mixture density functions which prone to create two distant groups in the data. In this situation a robust kernel density estimator is proposed to acquire a more efficient MI estimate in mixture distribution. The performance of the robust MI is investigated extensively by Monte Carlo simulations. The results of the study offer substantial improvement over the existing techniques

Robust multicollinearity diagnostic measures based on minimum covariance determinants approach

The classical multicollinearity diagnostic measures are not resistant to high leverage points since their formulation are based on eigen analysis of classical correlation matrix that is very sensitive to the presence of these leverages. The existing robust multicollinearity diagnostics also are not able to diagnose the variables which are collinear to each other. In this paper, we proposed robust multicollinearity diagnostic measures based on the Minimum Covariance Determination (MCD), which is a highly robust estimator of multivariate location and scatter. The results of numerical example and simulation study confirmed the merit of our new proposed robust multicollinearity diagnostic measures

Estimating bias and RMSE of indirect effects using rescaled residual bootstrap in mediation analysis.

It is a common practice to estimate the parameters of mediation model by using the Ordinary Least Squares (OLS) method. The construction of T statistics and confidence interval estimates for making inferences on the parameters of a mediation model, particularly the indirect effect, is usually are based on the assumption that the estimates are normally distributed. Nonetheless, in practice many estimates are not normal and have a heavy tailed istribution which may be the results of having outliers in the data. An alternative approach is to use bootstrap method which does not rely on the normality assumption. In this paper, we proposed a new bootstrap procedure of indirect effect in mediation model which is resistant to outliers. The proposed approach was based on residual bootstrap which incorporated rescaled studentized residuals, namely the Rescaled Studentized Residual Bootstrap using Least Squares (ReSRB). The Monte Carlo simulations showed that the ReSRB is more efficient than some existing methods in the presence of outliers

A comparison between classical and robust method in a factorial design in the presence of outlier

Analysis of Variance (ANOVA) techniques which is based on classical Least Squares (LS) method requires several assumptions, such as normality, constant variances and independency. Those assumptions can be violated due to several causes, such as the presence of an outlying observation. There are many evident in literatures that the LS estimate is easily affected by outliers. To remedy this problem, a robust procedure that provides estimation, inference and testing that are not influenced by outlying observations is put forward. A well-known approach to handle dataset with outliers is the M-estimation. In this study, both classical and robust procedures are employed to data of a factorial experiment. The results signify that the classical method of least squares estimates instead of robust methods lead to misleading conclusion of the analysis in factorial designs

Procedures of generating a true clean data in simple mediation analysis

Simulation study is very important in model validation. It is invaluable and versatile tool especially in statistical problems and modeling where analytical technique is inadequate. In fitting to a model, problems will raise when there exists one or more high-leverage points in the data set. Due to the fact that the presence high-leverage points are commonly occurred in models fitting, we propose a new algorithm in mediation analysis which guarantees clean data set without any high-leverage points. The new proposed algorithm employs the newly proposed Modified Diagnostic-Robust Generalized Potentials. By incorporating ModDRGP in the proposed algorithm has rectified the problem of having high leverage points in the generated clean data set, especially for mediation models. We found that in 10000 simulation runs, only about 31.14% of the cleangenerated dataset were obtained by direct simulation. The results also reveal that as the sample size increases, the percentage of obtaining direct clean dataset decreases