For estimating regression function we can use many proceedings. In this paper, we have chosen to apply scaling functions to the estimation of regression functions. When one knows many bivariate date with the values of two variables, in the goal to express a correlation between the two variables we use the regression function. The raw estimator of this function must be "smoothed out" in some way to get a final estimator. For this, we use the scaling functions, examples of such function being the Battle-Lemarié family and Daubechies family. After introducing several notions (multiresolution analysis, filter and projection of function onto approximation spaces), these are applied to obtain the estimators. In the last part, we present the algorithm for estimating nonparametric regression function through the scaling functions.nonparametric regression, scaling functions, filter, multiresolution analysis, ap-proximation space, estimator

Sorin MANOLE

English

Research Papers in Economics

Informatica Economică, nr. 1 (41)/2007  49
Application of the Scaling Functions to Nonparametric Regression 
 
Sorin MANOLE 
Constantin Brâncoveanu University Piteşti 
 
For estimating regression function we can use many proceedings. In this paper, we have cho-
sen to apply scaling functions to the estimation of regression functions. 
When one knows many bivariate date with the values of two variables, in the goal to express a 
correlation between the two variables we use the regression function. The raw estimator of 
this function must be “smoothed out” in some way to get a final estimator.  For this, we use 
the scaling functions, examples of such function being the Battle-Lemarié family and Daube-
chies family. After introducing several notions (multiresolution analysis, filter and projection 
of function onto approximation spaces), these are applied to obtain the estimators. In the last 
part, we present the  algorithm for estimating nonparametric regression function through the 
scaling functions. 
Keyworks: nonparametric regression, scaling functions, filter, multiresolution analysis, ap-
proximation space, estimator.  
 
ntroduction 
Th
bivar
e typical case is that one when we have 
iate data    and we 
want to use the data to quantify the relation-
ship between the two variables. The standard 
regression model is
() i i y x ,, n i , , 2 , 1 K =
( ) i i x i f y ε + = , 
(1), where the  n i , , 2 , 1 K = i ε  ‘s are inde-
pendent and identically distributed  ( )
2 , 0 σ N
f
x x
 
random variables. The primary interest is in 
estimating the regression function   from 
the data. In parametric regression, we assume 
a particular structure on the regression func-
tion    (polynomial, logarithmic, exponen-
tial etc.). Statistical inference involves de-
termining the parameter. Although 
parametric regression is used widely in 
practice, in many situations, one might be 
reluctant to choose a specific form of a 
model to fit a particular set of data. The field 
of nonparametric regression has developed to 
fit a curve to data without assuming any par-
ticular structure on the regression function . 
f
f
We assume that the values ’s are increas-
ingly ordered: 
i x
n x < < 2 < K 1
1 − − = i i i x x i
 and 
letd ,  n , , 3 , 2 K = . We define   
as mean of other values
1 d
i d
1
1 x
1
2
1 −
−
=
−
=
∑
=
n
x
n
d
d
n
n
i
i
1 1 d = − n x b =
b x x x x a n =
. 
If we note ,  , we obtain  0 x x a =
< < < < = K 2 1 0 , which means 
that the values of variable x lie on the inter-
val [ ] b a,
] 1 , 0
. Because we wish as these values to 
be placed in interval[ , we achieve a re-
scaling of these through linear transforma-
tion [ ] [ ] 1 , 0 , : → b a ϕ ,  ()
a b
a x
x
−
−
= ϕ . Using no-
tation ( ) i i x x ϕ = ′ ,i n , , 2 , 1 K = , we have 
1 0 2 1 0 = ′ < < ′ < ′ < ′ = n x x x x K
i x
] 1 , 0 i x
n d d
. For this rea-
sons, it will be assumed that the variables   
lie on the unit interval[ . Frequently,   
are equally spaced, which means 
thatd = = = K 2 1 . In this situation we ob-
tain
n
i
xi = ′ n i , , 2 , 1 K ,  =  (2). The raw estima-
tor of the regression function   is the piece-
wise constant function 
(3). If the val-
ues   are equally spaced, we have 
f
()
⎩
⎨
⎧
=
= < ≤
=
−
                                1    , 1
, , 2 , 1   ,    , ~ 1
x
n i x x x y
x f
i i i K
i x
()
⎪ ⎩
⎪
⎨
⎧
=
= < ≤ =
                               1    , 1
, , 2 , 1   ,
1
   , ~
x
n i
n
i
x
n
i-
y x f i K
R R → : f []R → b a, :
 (4). Farther 
we introduce some theoretical elements nec-
essary about scaling functions. 
Scaling functions 
A function   (f ) be-
longs to the square-integrable function space  
I Informatica Economică, nr. 1 (41)/2007  50 
() R
2 L  ( ), if both   and   are inte-
grable on  
[ b a L ,
2 ] f
2 f
R  (on ). Therefore, we have 
  . For two 
functions   ( ) we 
define inner product
[ b a, ]
]
() ∫
∞ +
∞ − ∞ <
 
 
2 dx x f () ⎟
⎠
⎞ ⎜
⎝
⎛ ∞ < ∫
b
a dx x f
 
 
2
() R
2 , L g f ∈ [ b a L g f , ,
2 ∈
()() ∫
∞ +
∞ − =
 
  , dx x g x f g f  
()() ⎟
⎠
⎞ ⎜
⎝
⎛ =∫
b
a dx x g x f g f
 
  , , by means him we 
obtain the norm of a function ( ) R
2 L f ∈  
[] ( ) b a L f ,
2 ∈ ,  f f f , = . A basis { }
Z ∈ j j f  
for   ( ) is a orthonormal basis 
if
() R
2 L [ b a L ,
2 ]
1 = j f ,  Z ∈ ∀j 0 , = k j f f ,  Z ∈ ∀ k j, , 
. The vectors of a set  k j ≠ ⊂ A ( ) R
2 L  
( ⊂ A [ b a L ,
2 ] ) span a subspace V  if 
 and we write 
. A sequence of subspaces 
⎭
⎬
⎫
⎩
⎨
⎧
∈ ∀ ∈ = ∑
∈
A a a V a
A a
a , /   R α α
() A Sp V = ( )
Z ∈ j j V  
of   ( ) form a multiresolution 
analysis if 1) 
; 2) 
,  
() R
2 L [ b a L ,
2 ]
K K ⊂ ⊂ ⊂ ⊂ ⊂ ⊂ − − 2 1 0 1 2 V V V V V
{} I
Z ∈
=
j
j V 0 =
∈ U
Z j
j V ( ) R
2 L  ( ); 3) 
,  ; 4) 
,  ; 5) There ex-
ists a function   such that the 
set
[ b a L ,
2 ]
j V f ∈ ⇔ () 1 2 + ∈ ⋅ j V f Z ∈ ∀j
0 V f ∈⇒ ()0 V k f ∈ − ⋅ Z ∈ ∀k
0 V ∈ Φ
{ } Z ∈ Φ k k   /   , j , where  () () k x x
j
j
k j − Φ = Φ 2 22
, , 
(5) constitutes an orthonormal basis 
for , . The fifth property show that 
Z ∈ ∀k
j V Z ∈ ∀j
= j V { } ( ) Z ∈ Φ k Sp k j   /   , . The set 
() { } Z ∈ − ⋅ Φ = Φ k k k   /   , 0 , obtained by apply-
ing integer translations, form a basis for  . 
The function   is called the scaling function 
since its dilates and translates constitute or-
thonormal bases for all   subspaces, which 
are simply scaled versions of . For any 
function   
0 V
Φ
j V
0 V
() R
2 L f ∈ [] ( ) b a L f ,
2 ∈ , we note 
 the projection  of a function   onto the 
subspace  . By virtue of the second prop-
erty, approximation   of function   
converges of this function , as 
f P
j f
j V
f P
j f
f f P
j →
∞ → j  in the   sense
2 L 0 → − f f P
j , as 
∞ → j . Because { } Z ∈ Φ k k j   /   ,  is a basis for 
, we get  j V ( )( )( x c x f P k j
k
k j
j
, , Φ = ) ∑
∈Z
 (6). For 
functions   with support compact 
(
f
( ) ( ) { } 0     p sup ≠ ∈ = x f / x f R ), it is easily 
seen that only finitely many sequence ele-
ments  { }
Z ∈ k k j c ,  will be non-zero. Since the 
set  { } Z ∈ Φ k k j   /   ,  is an orthonormal basis 
for , the scaling function coefficients can 
be computed 
j V
() () ∫
∞ +
∞ − Φ = Φ =
 
  , , , , du u u f f c k j k j k j (7). 
Far away, we propose to express the coeffi-
cients   of the approximation of   in 
space   in terms of the coefficients   of 
the approximation of   in space  . 
k j c , f
j V k j c , 1 +
f 1 + j V
Note that  0 V ∈ Φ  and therefore also   
since . Because 
1 V ∈ Φ
1 0 V V ⊂ { } Z ∈ Φ k k   /   , 1  is an or-
thonormal basis for  , there exists a se-
quence 
1 V
{ } Z ∈ k k h , which is called filter, such 
that  ( ) ( ) x h x
k
k k ∑
∈
Φ = Φ
Z
, 1 (8) and that the se-
quence elements may be written 
() () ∫
∞ +
∞ − Φ Φ = Φ Φ =
 
  , 1 , 1 , dx x x h k k k (9). This 
sequence is a square-summable sequence 
({ } Z ∈ k k h ∈ ( ) Z
2 l ), that is . Using 
(5) and (8), we may show that 
∞ < ∑
∈Z k
k h
2
( ) ( ) x h x
l
l j k l k j ∑
∈
+ − Φ = Φ
Z
, 1 2 , (10). Substituting 
this result into (7) we obtain 
∑
∈
+ − =
Z l
l j k l k j c h c , 1 2 , (11) relation which allow 
to write coefficients on   in terms of co-
efficients on  . The simplest scaling 
function is  . 
Applying a usual dilation and usual transla-
tions, we obtain  
f P
j
f P
j 1 +
() [ )
⎩
⎨
⎧ ∈
= Φ
                      otherwise    , 0
                        1 , 0    , 1 x
x
() () [ )
⎪ ⎩
⎪
⎨
⎧
+ ∈ = Φ
− −
                                           otherwise    , 0
                        1 2 , 2    , 2 2
, k k x x
j j
j
k j  Informatica Economică, nr. 1 (41)/2007  51
which show that approximation spaces   
consists of functions that are piecewise con-
stant, more exactly 
j V
() ( ) [ ) { } Z ∈
− k k
j   , 2 , R ∈ =
− k f / L f V
j
j 1 2 on  constant    piecewise   is      
2 +
. At the same time, according to (7), the coef-
ficients   are computed from the formula  k j c ,
()
()
∫
+
−
− =
1 2  
2  
2
, 2
k
k
j
k j
j
j du u f c . Since for  j  fixed, 
any   lie on the only one interval of 
type
R ∈ x
() [ ) 1 2 , 2 +
− − k k
j j , using (6) we obtain 
() () ()
()
∫
+
−
−
− =
1 2  
2   2
1 k
k j
j
j
j du u f x f P which show 
that this approximation of   in  f x is “mean” 
of all function values on the interval 
() [ ) 1 2 , 2 +
− − k k
j j . Another example of scaling 
function is the Battle-Lemarié family. The 
first member of the Battle-Lemarié family is 
identical to the scaling function defined be-
fore  () [ )
⎩
⎨
⎧ ∈
=
otherwise    , 0
1 , 0    , 1
1
x
x N . In constructing of 
the members of this family, the cardinal B-
spline functions are useful, which are piece-
wise polynomial functions, defined through 
convolution operator ∗ (for  ( ) R
2 , L g f ∈  we 
have: 
 () ).  ( ) ( ) ( ) ∫
∞ +
∞ − = ∗
 
  - dv v g v x f x g f
The first cardinal B-spline function is  ( ) x N1  
introduced before, and cardinal B-splines of 
higher order are defined recursively in terms 
of lower order B-splines by means of the 
convolution operator 
() ( ) () ( ) ∫ − = ∗ = − −
1  
0   1 1 1 du u x N x N N x N m m m , 
  K , 3 , 2 = n
It is interesting to note also that the function 
 is also the probability density func-
tion for the sum of   independent uniform 
 random variables. Therefore, by the 
Central Limit Theorem, as   gets large, the 
spline function 
() x Nm
m
( 1 , 0 )
m
( ) x Nm  approaches the prob-
ability density function of a normal random 
variable.  
The second member of the Battle-Lemarié 
family begins with the piecewise linear 
spline . But this function is not or-
thogonal to its integer translates and there-
fore the function   must be “orthogo-
nalized” to give an appropriate scaling func-
tion
() x N2
() x N2
( ) x N2 . Unfortunately, there is no closed 
form expression for the resulting scaling 
function, but it is given only in terms of its 
Fourier transform ()
2
cos 2 1
2
sin 4
2
3
2 2
2
ω
ω
ω
π
ω
+
= Φ
) . 
The function Φ is piecewise linear, it is not 
compactly supported, but it does decay expo-
nentially. Mallat gives general formulas for 
constructing scaling functions for the Battle-
Lemarié family, as well as for computing the 
associated filter{ } Z ∈ k k h . The resulting scaling 
functions will be polynomial splines. An-
other important family of scaling functions is 
the Daubechies family. They are compactly 
supported scaling functions. The derivation 
of this family represents quite a different ap-
proach. Thus, we begin with the filters, and 
then we derive from the filter coefficients the 
scaling function. Obvious, for compactly 
supportedΦ, only finitely many of the  ’s 
will be non-zero. Daubechies approach to 
constructing scaling functions begins with 
defining the trigonometric polynomial  
k h
() () ∑ ∑
∈Z k 2
1
∈
− − = =
Z k
k
ik
k k i k h e h m ω ω ω
ω sin cos
2
1
0  
The scaling functions are written in terms of 
this  ( ) ω 0 m  function, and, again, they can 
only be expressed in terms of their Fourier 
transforms () ( ∏
∞
=
− = Φ
1
0 2
2
1
j
j m ω
π
ω
) ) . There 
are several possibilities for the func-
tion ( ) ω 0 m , depending on the number of non-
zero elements in the filter. Clearly, the more 
non-zero filter elements make the smoother 
the resulting scaling functions.  
Estimation of nonparametric regression  
For nonparametric estimation of regression 
function it can us kernel smoothing, orthogo-
nal series estimation and scaling functions 
estimation. This paper will focus on only the 
scaling function version of these methods as 
applied to nonparametric regression, that is 
recovery of a regression function   as in (1).  fInformatica Economică, nr. 1 (41)/2007  52 
The estimator of regression function   will 
be the projection of raw estimator , defined 
in (3), onto the approximation space  . 
Therefore, according to (6), we have 
f
f
~
J V
() () () () x c x f P x f k J
k
k J
J
J , , ˆ ~ ˆ Φ = = ∑
∈Z
 (12). If in 
previous section we discussed the func-
tion , possibly with infinite sup-
port, farther we have the function  , evi-
dently square-integrable function on the in-
terval 
∈ f () R
2 L
f
~
[ ] 1 , 0 . The coefficients , which are 
estimation of “true” coefficients, according 
to (7) are computed by 
k J c , ˆ
() () ∫ Φ = Φ =
1  
0   , , ,
~
,
~ ˆ du u u f f c k J k J k J  (13). 
Substituting (13) into (12), using (3) and re-
arranging terms, we obtain 
() () () () () ∑ ∫ ∑ ∑∑ ∫
=∈ ∈=
− −
= ⎟
⎠
⎞
⎜
⎝
⎛
Φ Φ = Φ ⎟
⎠
⎞
⎜
⎝
⎛
Φ =
n
i
x
x
k
k J k J i k J
k
n
i
x
x k J i J
i
i
i
i
du x u y x du u y x f
1
 
  , , ,
1
 
  ,
1 1
ˆ
Z Z
() du u x E y
n
i
x
x J i
i
i
,
1
 
  1 ∑ ∫
=
−
     (14) 
where (15)  () ( ) ( ∑
∈
Φ Φ =
Z k
k J k J J x u u x E , , , )
The sum in (15) is finite because 
( ) [] 1 , 0
~
p sup = f  is compact.   defined 
above is a variable kernel. It is not apparent 
that the estimator   depends explicitly on 
the choice of parameter . A relatively large 
value of parameter   gives a narrow 
smoothing window, so the estimate of the 
function at 
J E
J f ˆ
J
J
x depends primarily on data near 
x. Thus, estimators for   large tend to be 
rather wiggly. Conversely, a smaller choice 
of parameter   gives a wider smoothing 
window, resulting in more averaging and a 
smoother estimate. Large values of    give 
estimators with smaller bias, but larger vari-
ance. Selecting a smaller parameter   will 
decrease the variance, but increase the bias. 
The developments above show that this esti-
mator is at the same time both an orthogonal 
series and a kernel estimator. For better em-
phasize of the correlation between the two 
variable 
J
J
J
J
x and , we need more estima-
tors , both for larger values of , and for 
smaller values of . Therefore, in the begin-
ning choose a scaling functionΦ, a larger 
value   of the parameter   and a smaller 
value   of the same parameter, and after-
wards we shall use the following algorithm: 
y
J f ˆ J
J
1 J J
0 J
1) We compute the estimated coeffi-
cients , which appear in writing of esti-
mator , using formula 
(16), according to (13). If 
it is necessary, the integral given in (16) is 
computed on the basis of a numerical method 
(Bessel, Hermite, Simpson, Romberg etc.). 
k J c , 1 ˆ
1
ˆ
J f
() du u y c
n
i
x
x k J i k J
i
i ∑ ∫
=
−
Φ =
1
 
  , ,
1
1 1 ˆ
2) Using (9) we compute the filter coeffi-
cients  () () ∫ − Φ Φ =
1  
0   2 2 dx k x x hk  (17) pos-
sibly on the basis of a numerical method. 
3) For M  large we compute values of the re-
gression function estimator   in the 
points
1
ˆ
J f
M
l
, according to (12) 
⎟
⎠
⎞
⎜
⎝
⎛ Φ = ⎟
⎠
⎞
⎜
⎝
⎛ ∑ M
l
c
M
l
f k J
k
k J J , , 1 1 1 ˆ ˆ ,  M l , , 1 , 0 K =  (18) and 
we represent it graphically. 
4) Using (11) we determine the coefficients 
which appear in writing of estimator ,  1 1
ˆ
− J f
∑ − − =
l
l J k l k J c h c , 2 , 1 1 1 ˆ ˆ  (19) 
5) With  1 1 − J  in place of   we take again 
the proceeding from step 3), until the value 
of this parameter gets equal to .  
1 J
0 J
Bibliography 
1. Antoniadis, A., Gregoire, G., McKeague, I. W. – Wavelet 
Methods for Curve Estimation, Journal of the American Sta-
tistical Association 89, pp. 1340-1353, 1994 
2. Daubechies, I. – Orthonormal Bases of Compactly Sup-
ported Wavelets, SIAM Journal on Mathematical Analysis 
24, pp. 499-519, 1993 
3.  Eubank, R. L. – Spline Smoothing and Nonparametric 
regression, Marcel Dekker, New York, 1988 
4. Ogden, R. T. – Essential Wavelets for Statistical Applica-
tions and Data Analysis, Birkhäuser, Boston, 1997 
5. Tătaru, G. – Serii Fourier. Transformata Laplace, Editura 
Economică Bucureşti, 2003 
6. Walter, G. G. – Wavelets and other Orthogonal Systems 
with Applications, CRC Press, Boca Raton, Florida, 1994 

Application of the Scaling Functions to Nonparametric Regression

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics