In this note we present r th order kernel density derivative estimators using canonical higher-order kernels. These canonical rescalings uncouple the choice of kernel and scale factor. This approach is useful for selection of the order of the kernel in a data-driven procedure as well as for visual comparison of kernel estimates.Derivative Estimation, AMISE

Christopher F. Parmeter

Daniel J. Henderson

English

Research Papers in Economics

CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE
ESTIMATION
DANIEL J. HENDERSON AND CHRISTOPHER F. PARMETER¤
Binghamton University and University of Miami
Abstract. In this note we present rth order kernel density derivative estimators using canonical
higher-order kernels. These canonical rescalings uncouple the choice of kernel and scale factor. This
approach is useful for selection of the order of the kernel in a data-driven procedure as well as for
visual comparison of kernel estimates.
Journal of Economic Literature Subject Classi¯cation: C14.
1. Introduction
The issue of kernel selection on the performance of density estimation is widely believed to have
little overall impact relative to that of the bandwidth. Yet, given that the bandwidth controls the
smoothing of the density estimator, using di®erent kernels (e.g., Epanechnikov vs. Gaussian) can
produce di®erent results. Here we generalize the canonical second-order kernel equivalence class
developed in Marron & Nolan (1989) to higher-order kernels.
1 The development of canonical higher
order and derivative kernels may be useful for data-driven selection of the bandwidth and the kernel
order simultaneously (Hall & Marron 1988).
The remainder of the paper is laid out as follows. Section 2 discusses the construction of the
higher-order equivalence class and provides the canonical kernel scalings. Section 3 gives a simple
set of illustrations to emphasize our results.
E-mail address: djhender@binghamton.edu,cparmeter@miami.edu.
Date: July 13, 2010.
Key words and phrases. Derivative Estimation, AMISE.
¤ Daniel J. Henderson (djhender@binghamton.edu) Department of Economics, State University of New York at
Binghamton and Christopher F. Parmeter (Corresponding Author, cparmeter@miami.edu) Department of Economics,
University of Miami.
1See Abadir & Lawford (2004) for a similar exercise using asymmetric kernels.
12 CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION
2. Estimators
The ºth-order, rth derivative kernel density estimator for f(r) (¢); based on an iid random sample
fX1;:::;Xng from f (¢), the probability density, is de¯ned as
(1) ^ f(r) (x) = n¡1h¡(1+r)
n X
i=1
k(r)
º;s
µ
x ¡ Xi
h
¶
;
where h is the bandwidth and k
(r)
º;s is the rth derivative of the ºth-order s-kernel (see (2) below). The
most common setup is º = 2 and r = 0 which constitutes a second-order kernel used to estimate
the density itself. The use of a higher-order kernel is typically undertaken to reduce the bias of the
density estimator.
There are also many cases where interest may lie in the rth derivative. For example, one may be
interested in looking for the location of modes (r = 1). Indirect interest in derivatives estimates
also exists. For example, estimation of the roughness of the second derivative (r = 2) of the density,
R(f(2) (¢)) =
R
f(2)(x)2dx; is required for plug-in bandwidth selection.
Marron & Nolan (1989) discuss an equivalence class of kernels for the case º = 2 and r = 0.
Here we generalize their results for the arbitrary º;rth case. We discuss estimation for the class of
kernels de¯ned as
(2) ks(u) =
(2s + 1)!!
2s+1s!
(1 ¡ u2)s1fjuj · 1g;
where the double factorial is de¯ned as (2s+1)!! = (2s+1)¢(2s¡1)¢¢¢5¢3¢1 (commonly known as
the odd factorial). As s ! 1, ks(u) ! e¡u2=2. Rescaling this particular case by 1=
p
2¼ delivers the
common Gaussian kernel, which we denote as kÁ(u). The Epanechnikov (s = 1), biweight (s = 2)
and triweight (s = 3) are also popular kernels from this class. Notice that as s increases, the kernel
possesses more derivatives and thus is `smoother'.
For the class of polynomial kernels of order s, a ºth-order s-kernel can be constructed as (Hansen
2005, Theorem 1)
kº;s(u) = Bº=2;s(u)ks(u);CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION 3
where
Bº=2;s(u) =
¡3
2
¢
º=2¡1
¡3
2 + s
¢
º=2¡1
(s + 1)º=2¡1
º=2¡1 X
j=0
(¡1)j ¡1
2 + s + º=2
¢
j x2j
j!(º=2 ¡ 1 ¡ j)!
¡3
2
¢
j
:
The notation (a)n =
¡(a+n)
¡(a) is Pochhamer's symbol, where ¡(a) =
1 R
0
ta¡1e¡tdt. See Wand &
Schucany (1990, Theorem 2.1) for a similar expression for higher order Gaussian kernels.
We ¯rst impose several generic properties on our kernel function involving the `moments' of the
kernel. Letting ·j(k) =
1 R
¡1
ujk(u)du; we say a kernel is of ºth-order if ·0(k) = 1, ·j(k) = 0 for
1 · j · º ¡ 1 and ·º(k) < 1. With symmetric kernels we have that ·`(k) = 0 for ` = 2j + 1,
i.e. all odd moments of our kernel are zero so that we may treat only the cases where º is even.
Requiring ·º (¢) to be ¯nite is necessary to obtain meaningful expressions for the bias of our kernel
density estimator.
As pointed out by Marron & Nolan (1989, pg. 197), a representative kernel has a best element.2
This kernel is known as the canonical kernel and it is such that it has exactly the same e®ect on
the squared bias and variance components which make up the asymptotic mean integrated squared
error (AMISE). Here we extend the results of Marron & Nolan (1989) to both higher-order
and derivative kernels. To establish our basis, we quantify the requisite amount of smoothing via
AMISE which one can easily show for the density derivative estimator in (1) to be
(3) AMISE( ^ f(r)(x)) ¼
R
¡
f(r+º) (¢)
¢
h2º·2
º (kº;s)
(º!)2 +
R
³
k
(r)
º;s
´
nh1+2r ;
provided h ! 0 and nh1+2r ! 1 and assuming that f (¢) is r+º times continuously di®erentiable.
We immediately notice that (3) quanti¯es the smoothing trade-o® since decreasing h results
in the ¯rst term (the squared bias) collapsing towards zero with the second term (the variance)
increasing. This produces an estimated curve with too much variation. Alternatively, allowing
h to increase clearly raises the ¯rst term while the second term shrinks, producing a curve that
is uninformative as it possesses almost no local variation since it averages over too large of a
neighborhood surrounding x. Moreover, regardless of the order of the derivative of interest, the
2For a given kernel one may scale it by any positive constant, thus, each scaled kernel is an element from the class of
kernels de¯ned by s and º in our language.4 CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION
bias always depends on the kernel, not its derivative. Alternatively, the variance depends directly
on the rth derivative of the kernel being used in estimation.
2.1. Optimal scaling. We now discuss a rescaled version of the kernel which decouples the impact
that the choice of kernel has on each component in (3). Following the setup of Marron & Nolan
(1989), we seek to rescale k
(r)
º;s so that k and h are separate in (3). This can be done by noting
that the kernel impacts the bias through ·2
º;s (kº;s); while it impacts the variance through R
³
k
(r)
º;s
´
.
By selecting the scale of the kernel so that these are equivalent, we see that we can equalize their
individual contributions to AMISE.3
Rescaling our kernel by ±,
k
(r)
º;s;±(¢) = k(r)
º;s(¢=±)=±r+1;
our optimal scale (±) is found by solving
·Z
xºkº;s;±(x)dx
¸2
=
Z ³
k
(r)
º;s;±(x)
´2
dx:
Integration by substitution yields
(4) ±0 =
h
R
³
k(r)
º;s
´i1=(1+2r+2º)
[·º (kº;s)]
¡2=(1+2r+2º) :
When r = 0 and º = 2; (4) gives the optimal scaling found in Marron & Nolan (1989).
With the choice ± = ±0, using the kernel k
(r)
º;s;±0(¢), delivers
(5) AMISE
³
^ f(r)(x)
´
¼ C
³
k
(r)
º;s;±0(¢)
´h
R
³
f(r+º) (¢)
´
h2º(º!)¡2 + (nh1+2r)¡1
i
;
where
C
³
k
(r)
º;s;±0(¢)
´
=
h
R
³
k(r)
º;s
´i2º=(1+2r+2º)
[·º (kº;s)]
(2+4r)=(1+2r+2º) :
These scalings are produced for various values of s, º, and r in Table 1. Here we ¯rst note the
obvious; the table shows fewer kernel functions (s) when the order of the derivative (r) increases
as some of our kernel functions only possess derivatives up to a given order. The relative di®erence
between estimates for a given kernel function can be seen by the relative values for ±. For example,
when r = 0, the e®ective smoothness of the Epanechnikov and Gaussian kernels di®er by a factor in
3An interesting extension would be to develop an equivalence class for the bias reducing kernels proposed in Mynbaev
& Martins-Filho (2010).CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION 5
excess of 2 for higher-order kernels and this di®erence increases with º. Note that this is when the
same bandwidth is used for smoothing. In contrast, we can surmise that comparisons between the
biweight and triweight kernels based on the same bandwidth would produce more closely related
density estimates for any order kernel.
3. Illustration
For both illustrative and comparison purposes, we consider the same data generating process
as Marron & Nolan (1989). Speci¯cally, we simulate 500 data points from a mixture density:
(0:7)Beta(4;8) + (0:3)Beta(40;20). We plot estimates using fourth-order kernels (º = 4).
Figure 1, panel (a) uses Gaussian (s = Á) and triweight (s = 3) kernels to estimate the underlying
density (r = 0), while panel (b) uses their respective canonical fourth-order kernels. Both panels
also provide vertically rescaled kernels centered at 0.5 to provide an equivalence (dotted lines).
Notice that in panel (a) the kernels do not appear to look similar and as such, even with the same
amount of smoothing, h = 0:11, the resulting curves are di®erent (due to the di®erence in the
kernels). In panel (b), we see that di®erences in the density estimates are harder to detect visually
given the use of canonical kernels. This is further buttressed by the vertically rescaled canonical
kernels at 0.5. The di®erences in these canonical kernels are di±cult to detect relative to panel (a).
In Figure 2 panel (a), we look at the estimates of the second-order derivatives of our density
(r = 2) using standard and canonical fourth-order kernels (º = 4), respectively. To make the panels
visually appealing we plot the vertically rescaled kernels using -400 as the x¡axis (again dotted
lines represent the kernels). We use the bandwidth h = 0:11 to construct our density derivative
estimates. We note that the 4th order, 2nd derivative density estimate using the triwieght kernel
is highly variable, suggesting that this bandwidth is too small (increased variance) whereas this
same bandwidth produces a viable estimate using the Gaussian kernel. However, the peaks and
the troughs are all underestimated suggesting perhaps that this bandwidth is too large (increased
bias). The di®erences in these estimated curves is easily gleaned by noting the di®erences in the
corresponding kernels at the bottom of each plot. Panel (b) provides the canonical forms of these
density estimates, where as expected, the di®erences in the estimated curves are di±cult to detect
visually.6 CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION
In sum, the use of canonical higher order derivative kernels may prove useful in a data driven
procedure which simultaneously selects the kernel order and the bandwidth since the use of canon-
ical kernels decouples the problem of kernel and bandwidth selection. This is an area of research
that has received little attention in the applied nonparametric literature.
References
Abadir, K. M. & Lawford, S. (2004), `Optimal asymmetric kernels', Economics Letters 83, 61{68.
Hall, P. & Marron, J. S. (1988), `Choice of kernel order in density estimation', Annals of Statistics 16(1), 161{173.
Hansen, B. E. (2005), `Exact mean integrated squared error of higher order kernel estimators', Econometric Theory
21, 1031{1057.
Marron, J. S. & Nolan, D. (1989), `Canonical kernels for density estimation', Statistics & Probability Letters 7(3), 195{
199.
Mynbaev, K. & Martins-Filho, C. (2010), `Bias reduction in kernel density estimation via Lipschitz condition', Journal
of Nonparametric Statistics 22, 219{235.
Wand, M. P. & Schucany, W. R. (1990), `Gaussian-based kernels', Canadian Journal of Statistics 18, 197{204.CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION 7
Table 1. Canonical kernel scalings (±): s refers to the kernel type. s = 1;2;3;Á
representing Epanechnikov, biweight, triweight and Gaussian kernels, respectively.
v = 2;4;6;8;10 refers to the order of the kernel funciton. r refers to the order of the
derivative of the function. r = 0;1;2 represents the density itself, and its ¯rst and
second order derivatives, respectively. The remaining values in the table represent
±, the canonical kernel scalings.
aaaa
s º 2 4 6 8 10
r = 0
1 1.7188 2.0165 2.0834 2.1021 2.1062
2 2.0362 2.2591 2.2694 2.2513 2.2302
3 2.3122 2.4788 2.4416 2.3913 2.3478
Á 0.7764 0.7214 0.6358 0.5686 0.5169
r = 1
2 1.9442 2.3658 2.4539 2.4607 2.4443
3 2.2103 2.5973 2.6410 2.6143 2.5736
Á 0.7559 0.7668 0.6959 0.6280 0.5717
r = 2
3 2.4189 2.8627 2.9335 2.9083 2.8585
Á 0.8415 0.8566 0.7818 0.7054 0.64058 CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION
Figure 1. The solid curve is the true underlying density function while the dashed
curve is the corresponding kernel density estimate. Both density estimates use 4th
order (canonical) kernels with bandwidth, h = 0:11. Vertically rescaled (canonical)
kernels appear as a dotted line in each panel.
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
4th Order Gaussian Kernel
x
d
e
n
s
i
t
y
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
4th Order Triweight Kernel
x
d
e
n
s
i
t
y
(a)
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
4th Order Canonical Gaussian Kernel
x
d
e
n
s
i
t
y
0.0 0.2 0.4 0.6 0.8 1.0
0
.
0
0
.
5
1
.
0
1
.
5
2
.
0
2
.
5
4th Order Canonical Triweight Kernel
x
d
e
n
s
i
t
y
(b)CANONICAL HIGHER-ORDER KERNELS FOR DENSITY DERIVATIVE ESTIMATION 9
Figure 2. The solid curve is the true underlying 2nd derivative of the density
function while the dashed curve is the corresponding 2nd derivative kernel density
estimate. Both derivative density estimates use 4th order (canonical) kernels with
bandwidth, h = 0:11. Vertically rescaled (canonical) kernels appear as a dotted line
in each panel.
0.0 0.2 0.4 0.6 0.8 1.0
−
4
0
0
−
2
0
0
0
2
0
0
4th Order Gaussian Kernel
x
2
n
d
 
D
e
r
i
v
a
t
i
v
e
 
o
f
 
D
e
n
s
i
t
y
0.0 0.2 0.4 0.6 0.8 1.0
−
4
0
0
−
2
0
0
0
2
0
0
4th Order Triweight Kernel
x
2
n
d
 
D
e
r
i
v
a
t
i
v
e
 
o
f
 
D
e
n
s
i
t
y
(a)
0.0 0.2 0.4 0.6 0.8 1.0
−
4
0
0
−
2
0
0
0
2
0
0
4th Order Canonical Gaussian Kernel
x
2
n
d
 
D
e
r
i
v
a
t
i
v
e
 
o
f
 
D
e
n
s
i
t
y
0.0 0.2 0.4 0.6 0.8 1.0
−
4
0
0
−
2
0
0
0
2
0
0
4th Order Canonical Triweight Kernel
x
2
n
d
 
D
e
r
i
v
a
t
i
v
e
 
o
f
 
D
e
n
s
i
t
y
(b)

Canonical Higher-Order Kernels for Density Derivative Estimation

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics