In this paper, new simple and multiple Nadaraya-Watson regression models are introduced by using a standard logistic kernel density. The statistical properties of the resulting regression functions are studied, and then the suggested models are applied to Egyptian fire accidents real data during the year 2020. Also, the suggested regression models are compared with other parametric and nonparametric regression models

M. Abo-El-Hadid, Samah

English

Arab Journals Platform

J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) 135Journal of Statistics Applications & ProbabilityAn International Journalhttp://dx.doi.org/10.18576/jsap/120113New Simple and Multiple Nonparametric RegressionModel Using a Standard Logistic Kernel with Applicationto Fire Accidents DataSamah M. Abo-El-HadidDepartment of Mathematics, Insurance and Applied Statistics, Faculty of Commerce and Business Administration, Helwan UniversityCairo, EgyptReceived: 15 Feb. 2022, Revised: 11 May 2022, Accepted: 13 May 2022Published online: 1 Jan. 2023Abstract: In this paper, new simple and multiple Nadaraya-Watson regression models are introduced by using a standard logistickernel density. The statistical properties of the resulting regression functions are studied, and then the suggested models are applied toEgyptian fire accidents real data during the year 2020. Also, the suggested regression models are compared with other parametric andnonparametric regression models.Keywords: Nadarya-Watson regression; standard logistic distribution; Kernel density estimation; Nonparametric regression; fireaccidents.1 IntroductionSuppose that Y and X are related by a simple regression model of the form:y = m(x)+ ∈ (1)where m(x) is the regression function and ∈ is a random error term. If we assume a certain structure on the regressionfunction m(x), then the regression model is called a parametric regression model. In the nonparametric regression model,the function m(x) is unknown and its estimation is the objective of predictive modelling. The regression function m(x)depends on the densities of the explanatory variable f(x), and the joint density function of both the dependent andindependent variable f(x,y). Rosenblatt [1] introduce a nonparametric estimator of the density function f(x), which calledthe kernel density estimator. Rosenblatt’s kernel density estimator is extended to the bivariate case by Epanechnikov [2].Nadaraya [3] and Watson [4] use the kernel estimators of Rosenblatt [1] and Epanechnikov [2] to estimate the regressionfunction m(x). Härdle and Gasser [5] introduce a robust non-parametric estimation for the regression function based onthe theory of M-estimation, and studied the statistical properties of the resulting estimator. Michels [6] suggested usingasymmetric kernel functions to estimate the regression function. He found that the asymmetric kernel techniquesoutperform the usual symmetric kernel methods and lead to better predictions according to real data application. Aalen[7] developed the kernel regression model in the case of survival analysis. He also uses the bootstrap replications tojudge the accuracy of the cumulative regression plots. Brown and Chen [8] use the beta family of density functions askernels function in estimating the regression function. The method they used is a generalization of Bernsteinpolynomials. Peristera and Kostaki [9] use kernel regression estimators in graduating age-specific mortality data forFrance, Japan, and Sweden. They found that The Gasser-Muller estimator is superior to the other kernels estimators.Park et. Al. [10] investigate the properties of L2 boosting with kernel regression estimates. They found that L2 boostingreduces the bias of the estimate, while it does not deteriorate the order of the variance. Martın-Baosa et. Al. [11] used thekernel estimator to estimate the utility function of the alternatives for the decision-maker whose stochastic part has amultinomial logistic model. In this paper, we introduce a simple nonparametric standard logistic regression model and a∗ Corresponding author e-mail: s aboelhadid@yahoo.comc© 2023 NSPNatural Sciences Publishing Cor.136 S. M. Abo-El-Hadid: New simple and multiple nonparametric logistic regressionmultiple regression model. The rest of this paper is organised as follows: In section 2 the kernel regression model isoverviewed, while in section 3, the new simple and multiple nonparametric regression model are introduced, and itsstatistical properties are obtained. In section 4 the new model is applied to Egyptian fire accidents real data for the simpleand multiple regression models.2 the nonparametric regression modelNote that the simple regression function can be expressed as a function in both f (x) and f (x,y) as follows:m(x) = E (Y |X ) =∫y f(Y |X ) (2)∴ m(x) =∫yf (x,y)f (x)dy (3)The Rosenblatt’s kernel density estimator for the density f (x) takes the following form:f̂ (x) =1nhn∑i=1K(x− xih) (4)where n is the sample size; K() and h are the kernel function and the bandwidth respectively, where the kernel functionK() is assumed to be a density function.Epanechnikov [2] extends the Rosenblatt’s kernel estimator to the bivariate case, which takes the form:f̂ (x,y) =1nn∑i=11hdK1(x− xih)K2(y− yid)(5)Nadaraya [3] and Watson [4] suggested replacing the previous densities in (3) by their kernel density estimators in (4) and(5), thenm̂(x) =1f̂ (x)∫1nhdn∑i=1K(x− xih)K(y− yid)y dy (6)=∑ni=1 K(x−xih)f̂ (x)∫1nhdK(y− yid)y dy (7)Let[u =y− yid, c =∑ni=1 K(x−xih)f̂ (x)] (8)substituting from (8) into (7), then:m̂(x) = c[∫1nhK(u)(ud+ yi)du]= c[∫dnhuK(u)du+∫1nhyiK(u)du]Note that,∫ ∞−∞ uK(u)du = 0, then[m̂(x) =1nn∑i=1{Kh(x− xi)n−1 ∑ni=1 Kh(x− xi)}yi =1nn∑i=1Li(x)yi] (9)The above estimator in (9) is biased estimator, to prove that the estimated regression function in (9) can be rewritten as:m̂(x) =1n ∑ni=1 Kh(x− xi)yi1n ∑ni=1 Kh(x− xi)=r̂(x)f̂ (x)(10)c© 2023 NSPNatural Sciences Publishing Cor.J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) / www.naturalspublishing.com/Journals.asp 137thenm̂(x)−m(x) =[r̂(x)f̂ (x)−m(x)]·[f̂ (x)f (x)+{1−f̂ (x)f (x)}]=r̂(x)−m(x) f̂ (x)f (x)+{m̂(x)−m(x)}·f (x)− f̂ (x)f (x)(11)The second term in (11) can be ignored, thenm̂(x)≈r̂(x)−m(x) f̂ (x)f (x)+m(x)≈r̂(x)−m(x){f̂ (x)− f (x)}f (x)∴ E(m̂(x)) =1f (x)[E(r̂(x))−m(x)E{ f̂ (x)− f (x)}] (12)Note that, Bias( f̂ (x))≈ h22f′′(x)σ2K , where:σ2K =∫u2K(u)du (13)E (r̂(x)) =1nhE[n∑i=1K(x− xih)yi]=1h∫ ∫K(x− th)y f(t,y) dt dy∫y f (y, t)dy = m̂(x) f (t)thenE(r̂(x)) =1h∫K(x− th)m̂(x) f (t)dtUsing Taylor’s expansion of ̂m(x+ uh), f (x+ uh), yields:E (r̂(x))≈ f (x)m(x)+h22f (x)m′′(x)σ2K + · · ·+ h2 f′(x)m′(x)σ2K +h22m(x) f′′(x)σ2K (14)thenE(m̂(x))≈ m(x)+h22m′′(x)σ2K + h2m′(x)f′(x)f (x)σ2KThen the bias of the estimated regression function is as follows:Bias(m̂(x))≈h22σ2K{m′′(x)+ 2m′(x)f′(x)f (x)}(15)And the variance of the estimated regression function is as follows:Var(m̂(x))≈σ2(x)nhf(x)∫K2(u)du (16)c© 2023 NSPNatural Sciences Publishing Cor.138 S. M. Abo-El-Hadid: New simple and multiple nonparametric logistic regressionWhere σ2(x) =Var (Y |X = x ).By the same way as in (3), the multiple Nadaray-Watson regression takes the following form:m(x) =∫f (y,x1, . . . ,xp)yf (x1, . . . ,xp)dy (17)And using the kernel density estimators, the estimated multiple regression model is:m̂(x) =∑ni=1[∏pj=1 K(x j−xijh)]yi∑ni=1[∏pj=1 K(x j−xijh)] (18)In the next section, we introduce the suggested simple and multiple regression models using the univariate andbivariate standard logistic kerel density estimators.3 The new simple and multiple nonparametric regression modelsIn this section, the new simple and multiple nonparametric regression models with Logistic kernel density estimatorsare introduced. Abo-El-Hadid [12] used the standard logistic distribution as a kernel function to introduce the univariatelogistic kernel density estimator. This estimator takes the form:f̂ (x) =1nhn∑i=1e−(x−xih)[1+ e−(x−xih)]2 , −∞ ≤ x ≤ ∞ (19)with the following statistical properties:E(u) =∫ ∞−∞uK(u)du = 0 (20)σ2K = var(u) =∫ ∞−∞u2K(u)du =π23(21)∫ ∞0K2(u)du =16(22)And the optimal bandwidths which minimize the integrated mean squared error (IMSE)hopt =[nπ463]− 15(23)In 2021 Abo-El-Hadid [13] extended this univariate logistic kernel density estimator in (19) to the bivariate case. Thebivariate logistic kernel density estimator introduced by Abo-El-Hadid (2021), and its statistical properties is as follows:f̂ (x,y) =1nh2n∑i=1e−(x−Xih)[1+ e−(x−Xih)]2 •e−(y−yih)[1+ e−(y−yih)]2 (24)with the following statistical properties:Bias[f̂ (x,y)]≈π4h218[∂ 2∂x21f (x,y)+∂ 2∂x22f (x,y)](25)Var[f̂ (x,y)]≈f (x,y)36nh2(26)c© 2023 NSPNatural Sciences Publishing Cor.J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) / www.naturalspublishing.com/Journals.asp 139IMSE[f̂ (x,y)]=136nh2+∫∫ ∞−∞π8h4324[∂ 2 f (x,y)∂x21+∂ 2 f (x,y)∂x22]2dx dy (27)And the optimal bandwidths which minimize the IMSEhopt =[3214195nπ8]− 16(28)Then the new simple regression function takes the following form:m̂(x) =∑ni=1e−(x−xih)[1+e−(x−xih)]2 yi∑ni=1e−(x−xih)[1+e−(x−xih)]2(29)The bias of this new regression function is obtained by substituting (19) into (15). then:Bias(m̂(x))≈h2π26{m′′(x)+ 2m′(x)f′(x)f (x)}(30)Abo-El-Hadid [12] suggested replacing the unknown term f′(x) in (30) by the standard logistic density as a referencedistribution. thenf′(x) =2e−2x(1+ e−x)3−e−x(1+ e−x)2(31)Then the estimated biased of the estimated regression function is as follows:B̂ias(m̂(x))≈h2π26{m′′(x)+2m′(x)f (x)[2e−2x(1+ e−x)3−e−x(1+ e−x)2]}(32)Substituting (22) into (16) yields that the variance of the estimated regression function is:Var(m̂(x))≈σ2(x)6nh f (x)(33)Then the asymptotic mean squared error of the new regression function is as follows:MSE(m̂(x))≈16nhσ2(x)f (x)+[h2π26{m′′(x)+2m′(x)f (x)[2e−2x(1+ e−x)3−e−x(1+ e−x)2]}]2By the same way, the estimated multiple standard logistic regression model with two explanatory variables is obtainedby substituting (24), into (9), so the suggested multiple regression function takes the following form:m̂(x) =∑ni=1e−(x1−xi1h)[1+e−(x1−xi1h)]2e−(x2−xi2h)[1+e−(x2−xi2h)]2 yi∑ni=1e−(x1−xi1h)[1+e−(x1−xi1h)]2e−(x2−xi2h)[1+e−(x2−xi2h)]2(34)c© 2023 NSPNatural Sciences Publishing Cor.140 S. M. Abo-El-Hadid: New simple and multiple nonparametric logistic regressionThen the bias and variance of the multiple regression function are as follows:Bias(m̂(x))≈12σ2κ trHT HmH +σ2κf (){∇TmHHT ∇ f } (35)Var(m̂(x))=1n ·det(H)·σ2(x)f ()∫κ2(u)du (36)Where (H) is a diagonal matrix of bandwidth;(H m) is the second derivative matrix of the function m̂(x); and ∇is the first derivatives.4 Real Data ApplicationIn this section the new simple and multiple nonparametric regression models are applied to Egyptian fire accidents realdata. The data used in this section is obtained from the annual report of fire accidents in Egypt 2020 (issue April 2021)which published by the Central Agency for Public Mobilization and Statistics (CAPMS).At the governorate level, Cairo comes first in terms of fire accidents with 13.3% of the total fire accidents duringthe year 2020, followed by Giza Governorate with 7.4%, and in the last rank North Sinai Governorate with 0.2%. alsoat the governorate level, Cairo comes first in terms of fire points with 11% of the total fire points, followed by SharkiaGovernorate with 7%, and in the last rank red sea and North Sinai Governorates with 1.1%.4.1 Application of simple regression modelIn this subsection, we study the relationship between the percentage fire stations (% fire points) an independent variable,and the percentage of fire accidents (% fire accidents) as a dependent variable.The following figure illustrate percentage of both fire accidents and fire points in each governorate during 2020.Figure (2) gives the scatter plot of the percentage of fire accidents and fire points in each Egyptian governorate during2020.The parametric linear regression model is estimated and the regression line with the scatter plot is given in figure(3): Then the new regression model using the logistic kernel is applied to the fire data and then compared with theparametric one; and the nonparametric regression model using the uniform kernel. Figure (4) and table (1) compares thethree regression models.From figure (4), it appears that the new regression model (in green) fits the fire data better than both the parametricand nonparametric uniform regression models (in red).Table 1: Comparison between parametric regression; and nonparametric regression models with logistic kernel and uniform kernelcriteria Parametric linear Nonparametric uniform Nonparametric logisticResidual standard error 1.569 1.567 1.346749R-squared 0.7118 0.7125 0.7822888Table (1) shows that new regression model for fire data outperform the parametric and the uniform nonparametricregression models according to the residuals standard error and R-squared criteria, and the worst model is the parametricone.4.2 Application of multiple regression modelIn this subsection, we study the relationship between percentage of deaths due to fire accidents as a dependent variableand the percentage of fire stations, and percentage of fire accidents as independent variables.Figure (5) gives a 3D scatter plot, of the dependent and independent variables.Figure (6) gives the scatter plot matrix and the estimated densities of the dependent and independent variables.Then the suggested multiple regression model using the logistic kernel is applied to the fire data and then comparedwith the parametric linear regression model. Figure (7) and table (2) compares the parametric and nonparametricregression models.Figure (7) and table (2), show that the suggested multiple regression model fits the fire data better than the parametriclinear regression model.c© 2023 NSPNatural Sciences Publishing Cor.J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) / www.naturalspublishing.com/Journals.asp 141Fig. 1: % of total number fire accidents and % of total fire stations in governorates in 2020Fig. 2: scatter plot of percentages fire accidents and percentages fire stations in 2020c© 2023 NSPNatural Sciences Publishing Cor.142 S. M. Abo-El-Hadid: New simple and multiple nonparametric logistic regressionFig. 3: scatter plot with the parametric regression line for percentage fire accidents and percentage of fire stations in 2020Fig. 4: scatter plot; parametric regression line; and nonparametric regression curves with logistic kernel (in green) and uniform kernel(in red)c© 2023 NSPNatural Sciences Publishing Cor.J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) / www.naturalspublishing.com/Journals.asp 143Fig. 5: 3Dscatter plot of% of deaths due to fire accidents and % fire accidents and % fire stations in 2020Fig. 6: scatter plot matrix and density estimate for the dependant and independent variables of fire data5 ConclusionIn this paper, a new regression model was derived using nonparametric logistic density functions presented by Abo-El-Hadid (2018) and (2021). The statistical properties of the presented regression model were also derived, which are bias,variance, and mean squares of errors. Finally, the suggested model was applied to analyze the fire accidents real datain Egyptian governorates during 2020, and we found that the nonparametric regression models overcome the parametriclinear regression model.c© 2023 NSPNatural Sciences Publishing Cor.144 S. M. Abo-El-Hadid: New simple and multiple nonparametric logistic regressionFig. 7: scatter plot; parametric regression line (solid line); and nonparametric regression curves with logistic kernel (dashed line)Table 2: Comparison between multiple parametric regression; and nonparametric regression models with logistic kernelcriteria Parametric linear Nonparametric logisticResidual standard error 14.970 12.035R-squared 0.04439 0.295AcknowledgementThe authors are grateful to the anonymous referee for a careful checking of the details and for helpful comments thatimproved this paper.References[1] M.Rosenblatt, Remarks on Some Nonparametric Estimates of Density Function. The Annals of Mathematical Statistics 27, 832-837(1956).[2] V.A Epanechnikov, Nonparametric Estimation of Multivariate Probability Density. Theory of Probability and its Application 14,153-158 (1969).[3] E. A. Nadaraya, On Estimating Regression. Theory of Probability and its Application 9, 141-142 (1964).[4] G. S. Watson, Smooth Regression Analysis, Sankhyã 26, 359-372 (1964).c© 2023 NSPNatural Sciences Publishing Cor.J. Stat. Appl. Pro. 12, No. 1, 135-145 (2023) / www.naturalspublishing.com/Journals.asp 145[5] W. Härdle, and T. Gasser, Robust Non-Parametric Function Fitting. Journal of the Royal Statistical Society: Series B(Methodological) 46, 42-51 (1984).[6] P.Michels, Asymmetric Kernel Functions in Non-Parametric Regression Analysis and Prediction. Journal of the Royal StatisticalSociety: Series D (The Statistician) 41, 439-454 (1992).[7] O.O Aalen, Further results on the non-parametric linear regression model in survival analysis, Statistics in medicine 12, 1569-1588(1993).[8] B.M Brown and S.X Chen, Beta-Bernstein Smoothing for Regression Curves with Compact Support,Scandinavian journal ofstatistics 26, 47-59 (1999).[9] P. Peristera, and A. Kostaki, An Evaluation of The Performance of Kernel Estimators for Graduating Mortality Data. Journal ofPopulation Research 22, 185-197 (2005).[10] B.U. Park, Y.K. Lee and S. Ha, L2 boosting in kernel regression. Bernoulli 5, 599-613 (2009).[11] J.A. Martın-Baosa, R. G. Rodenas., M. L. Garcia, iscrete choice modeling using Kernel Logistic Regression 47, 457–464 (2020).[12] S. M. Abo-El-Hadid, Logistic Kernel Estimator and Bandwidth Selection for Density Function, International Journal ofContemporary Mathematical Sciences 13, 279 – 286 (2018).[13] S. M. Abo-El-Hadid, A Suggested Nonparametric Bivariate Logistic Density Estimator with application on the productivity ofEgyptian wheat during 2019/2020,Journal of Mathematics and Statistics, 17, 44-49 (2021).[14] T. Gasser and H. G. Mülle, Kernel estimation of Regression Functions, in Gasser and Rosenblatt (eds), Smoothing Techniques forCurve Estimation. Heidelberg, Springer Verlag (1979).[15] W. Härdle, M. Müller, S. Sperlich, and A. Werwatz. . Nonparametric and Semi-parametric Models: An Introduction. New York,Springer-Verlag. (2004).c© 2023 NSPNatural Sciences Publishing Cor.

New Simple and Multiple Nonparametric Regression Model Using a Standard Logistic Kernel with Application to Fire Accidents Data

https://digitalcommons.aaru.edu.jo/cgi/viewcontent.cgi?article=1539&context=jsap

New Simple and Multiple Nonparametric Regression Model Using a Standard Logistic Kernel with Application to Fire Accidents Data

Abstract

Similar works

Full text

Available Versions

Arab Journals Platform