Traditional literature on impact evaluation often deals with binary programs. In reality, a program can provide different amounts of treatment for people, and it is regarded as a continuous variable. This paper discusses impact evaluation of a continuous program using matching methods and illustrates how to estimate impact of foreign remittances on per capita expenditure in Vietnam.Treatment effect, program impact, continuous program, matching, conditional independence

Nguyen Viet, Cuong

English

Research Papers in Economics

  1 
 
 
Estimating Impact of a Continuous Program under a 
Conditional Independence Assumption 
 
 
 
 
 
 
Nguyen Viet Cuong
1 
 
 
 
 
           Abstract 
 
Traditional literature on impact evaluation often deals with binary programs. In reality, a 
program can provide different amounts of treatment for people, and it is regarded as a 
continuous variable. This paper discusses impact evaluation of a continuous program 
using matching methods and illustrates how to estimate impact of foreign remittances on 
per capita expenditure in Vietnam. 
 
 
 
Keywords:  Treatment  effect,  program  impact,  continuous  program,  matching, 
conditional independence. 
JEL classification: C14; C15; H43; J41 
 
                                                       
1 Development Economics Group, Mansholt Graduate School, Wageningen University, the Netherlands.  
Address: Hollandsweweg 1, 6706 KN Wageningen, the Netherlands. 
E-mail: c_nguyenviet@yahoo.com    2 
1. Introduction  
 
The main objective of impact evaluation is to assess the extent to which a program has 
changed outcomes of subjects.
2 The average impact of a program on a group of subjects 
is defined as the difference between their outcome in the presence of the program and 
their outcome in the absence of the program. However, for each subject, we are not able 
to observe these two potential outcomes at the same time. This missing data problem can 
be  solved  if  an  assumption  of  conditional  independence  of  treatment  and  potential 
outcomes  holds  (Rubin,  1977).  Under  this  assumption,  the  program  impact  can  be 
estimated by OLS regression or matching methods. The idea of the matching method is 
to  compare  outcomes  of  participants  and  non-participants  who  have  the  same 
conditioning variables.  
The  literature  of  program  impact  evaluation  often  deals  with  impact  of  binary 
programs. In reality, a program can provide different amounts of treatment for people, 
and  it  can  be  regarded  as  continuous  instead  of  binary.  This  paper  discusses  impact 
evaluation  of  a  continuous  program  using  matching  methods  and  illustrates  how  to 
estimate impact of foreign remittances on per capita expenditure in Vietnam. 
   The paper is organized into five sections as follows. The second section presents 
the matching method in impact evaluation of a single binary program. The third section 
discusses  the  matching  method  in  the  case  of  a  continuous  program.  Results  of  the 
empirical example are presented in the fourth section. Finally the fifth section concludes.  
 
2. Impact Evaluation of a Binary Program    
 
Two popular parameters in program impact evaluation are the Average Treatment Effect 
(ATE), and the Average Treatment Effect on the Treated (ATT), which are expressed as 
follows:
3   
( ) ) X | Y ( E ) X | Y ( E ) X | ( E ATE X 0 1 − = = ∆ ,              (1) 
( ) ) D , X | Y ( E ) D , X | Y ( E ) D , X | ( E ATT X 1 1 1 0 1 = − = = = = ∆ ,          (2) 
                                                       
 
3 There are other parameters such as local average treatment effect, marginal treatment effect, or even 
effect of “non-treatment on non-treated” which measures what impact the program would have on the non-
participants if they had participated in the program, etc.    3 
where  D is a binary variable of program participation, X are exogenous conditioning 
variables,  0 Y  and  1 Y  are potential outcomes without and with the program, respectively.
4 
The  identification  assumption  for  matching  is  the  conditional  independence 
assumption as follows:  
Assumption 1:  X D Y , Y ⊥ 1 0                      (A.1) 
The assumption states that conditioning on X,  1 0   and   Y Y  are independent of D. Under this 
assumption,  ) X ( ATE and ) X ( ATT can be identified as follows:  
) D , X | Y ( E ) D , X | Y ( E ) X | Y ( E ) X | Y ( E ATE ) X ( 0 1 0 1 0 1 = − = = − = ,       (3) 
) D , X | Y ( E ) D , X | Y ( E ) D , X | Y ( E ) D , X | Y ( E ATT ) X ( 0 1 1 1 0 1 0 1 = − = = = − = = .   (4) 
The  matching  method  estimates  these  parameters  by  comparing  the  outcome  of  the 
participants and the outcome of a so-called comparison group who do not participate in 
the program but have the X variables identical to those of the participants.  
A difficulty in the matching method is to find matched non-participants for the 
participants when there are many the X variables. In the literature of impact evaluation, 
three  widely-used  methods  to  match  non-participants  with  participants  are 
subclassification  (see,  e.g., Cochran and Chambers, 1965; Cochran, 1968), covariate 
matching  (Rubin,  1979,  1980),  propensity  score  matching  (Rosenbaum  and  Rubin, 
1983).    
 
3 Impact Evaluation of a Continuous Program  
 
In reality, a program can provide different amounts of treatment for people. Denote a 
continuous program by d whose value is equal to the treatment level that the program can 
provide for people. For those who do not take the program, d is equal to 0. We can be 
interested in impact of the program at a range (interval) of the treatment  ] L  , [L 2 1 : 
( ) ( ) ( ) X Y E X Y E ATE d L L d L L d X 0 ] , [ ] , [ , 2 1 2 1 = ∈ ∈ − = ,                (5) 
( ) ( ) ( ) ] , [ , ] , [ , 2 1 0 2 1 ] , [ ] , [ , 2 1 2 1 L L d X Y E L L d X Y E ATT d L L d L L d X ∈ − ∈ = = ∈ ∈ .        (6) 
                                                       
4 Y0 and Y1 can be vectors of outcomes, but for simplicity let’s consider a single outcome of interest.    4 
If d is real value and bounded, we can denote the range of the d variable by  ] L   , [L max min . 
We  can  divide  d  into 1 + n   mutually  exclusive  and  successive  intervals 
] L ; (L   ],..., L   ; (L   ], L ; [L max n 2 1 1 min .  
Suppose that we can write the equations of the potential outcomes as follows: 
] L , L [ d max min ∉   ⇒  0 0 0 0 ε β α + + = X Y , 
] L , L [ d min 1 ∈   ⇒  1 1 1 1 ε β α + + = X Y , 
.... 
] L , L ( d max n ∈   ⇒  1 1 1 1 + + + + + + = n n n n X Y ε β α . 
The observed outcome is then written in terms of the potential outcomes: 
{ } { } { } ] L , L ( d I ) Y Y ( ... ] L , L ( d I ) Y Y ( ] L , L [ d I ) Y Y ( Y Y max n n min min ∈ − + + ∈ − + ∈ − + = 0 2 0 2 1 0 1 0
                            (7) 
Where I{} is an indicator function that is equal to 1 if the value of {} is true, 0 otherwise.  
Substitute the equations of the potential outcomes into (7), we get: 
[ ] { } [ ] { }
[ ] { }
{ } { } { } [ ] ) ( ] L , L ( D I ... ) ( ] L , L ( D I ) ( ] L , L [ D I         
] L , L ( D I ) ( X ) ( ...         
] L , L ( D I ) ( X ) ( ] L , L [ D I ) ( X ) ( X Y
n max n min 0
max n n n
min
0 1 0 2 2 1 0 1 1
0 1 0 1
2 1 0 2 0 2 1 0 1 0 1 0 0
ε ε ε ε ε ε ε
β β α α
β β α α β β α α β α
− ∈ + + − ∈ + − ∈ + +
∈ − + − + +
∈ − + − + ∈ − + − + + =
+
+ +
                            (8) 
Similar  to  the  case  of  a  single  binary  program,  impact  of  the  program  at 
] L , L ( L d i i 1 + ∈ =  can be identified under the conditional independence assumption. 
Assumption 2: Conditional independence between the potential outcomes and treatment 
levels: 
{ }X ] L , L ( d I   Y , Y i i i 1 0 + ∈ ⊥   j , i   ∀                  (A.2) 
Then,  impact  of  the  treatment  at  the  interval  ] L , L ( i i 1 + can  be  estimated  either  by 
parametric  methods,  e.g.,  running  an  OLS  regression  on  equation  (8)  with  dummy 
variables di indicating { } ] L , L ( d I   i i 1 + ∈ , or by non-parametric methods, e.g., the matching 
method by matching those who receive the treatment of level ] L , L (   i i 1 +  with those who 
do not receive the treatment (i.e. d = 0).  
In reality, there can be a problem in estimating the program impact in the region of 
no  data  on  the  program  treatment.  To  infer  the  program  impact  to  the  region  of  no 
treatment  level,  we  can  assume  a  functional  form  of  the  expected  program  impact   5 
conditional  on  X  and  d.  Recall  that  ATE(X)  and  ATT(X)  at  a  given  treatment  level 
] L , L ( L d i i 1 + ∈ =  using a partition of n intervals are: 
) ( X ) ( ATT ATE i i ) n ], L , L ( d , X ( ) n ], L , L ( d , X ( i i i i 0 1 0 1 1 1 β β α α − + − = = + + ∈ ∈ + + .        (9) 
The conditional parameters depend on X, d, and the number of partitioned intervals that 
is indicated by n. Thus we can write: 
) , ( ) , , ( ) , , ( d X f ATT ATE n n d X n d X = = ,                  (10) 
where  () fn  are real functions defined on domain of X and d.  d) (X, fn  has an important 
property that  0 0) d (X, fn = = , i.e. the impact of the program equals to 0 when there is no 
treatment. Substitute  d) (X, fn  into equation (8), we have:
5 
ε β α + + + = ) , ( 0 0 d X f X Y n                     (11) 
where:  
{ } { } { } [ ] ) ( ] L , L ( d I ... ) ( ] L , L ( d I ) ( ] L , L [ d I n max n min 0 0 1 0 2 2 1 0 1 1 ε ε ε ε ε ε ε ε − ∈ + + − ∈ + − ∈ + = +
If assumption (A.2) holds for any interval of the treatment, ε has the traditional property 
that  0 = ) d , X | ( E ε . 
To define the average treatment effect at any arbitrarily small interval, let n go to 
infinity, and the interval,  ] L   , [L max min , is divided into infinite sub-intervals as a result. 
Further, we can assume that functions  ) d , X ( fn  converge to a function  ) d , X ( f as n goes 
to infinity, i.e.:  
) d , X ( f lim ) d , X ( f n
n ∞ →
= .                    (12) 
Then, (11) becomes:  
ε β α + + + = ) , ( 0 0 d X f X Y .                    (13) 
Once the functional form of  d) f(X,  is specified, its parameters can be estimated by two 
ways. The first way is to run an OLS regression of Y on X and D using a function of Y, 
e.g., as in (13). The second way is to estimate program impact for all the participants by 
the matching method, and then to run an OLS regression of these impact estimates on X 
and D. The second method can be regarded as a semi-parametric method. Compared with 
the first approach, it is more robust in the sense it requires a functional form of  ) , ( d X f , 
while the first approach requires functional forms of both Y and ) , ( d X f .  
                                                       
5 Note that  { } { } { } { } 1 ] L , L [ D I ] L , L ( D I ... ] L , L ( D I ] L , L [ D I max min max n 2 1 1 min = ∈ = ∈ + + ∈ + ∈ .   6 
We can also define the marginal treatment effect conditional on X and d as follows: 
d
) d , X ( f
d
ATE
MTE
) d , X (
) d , X ( ∂
∂
=
∂
∂
= ,                  (14) 
which measures how the conditional average treatment effect changes as the treatment 
variable  changes. In other words, MTE(X,d) measures the additional amount of outcome 
that one can gain when receiving an additional treatment level. 
 
4. Empirical Example  
 
Vietnam  receives  a  large  amount  of  foreign  remittances  annually.  The  total  foreign 
remittances  were  nearly  4  billion  USD  in  2006.  Foreign  remittances  are  expected  to 
increase  household  consumption.  To  measure  impact  of  foreign  remittances,  we  use 
Vietnam  Household  Living  Standard  Survey  in  2004.  The  survey  is  conducted  by 
General Statistical Office of Vietnam with technical support of World Bank. The number 
of households covered by the survey is 9188.  
If we estimate the impact by OLS regression, we can assume that outcome has the 
following functional form: 
ε θ γ β α + + + + =
2 d d X Y ,                    (15) 
where Y is per capita expenditure or log of per capita expenditure, X are household 
characteristics,  d  is  amount  of  foreign  remittance  received  by  households.  The  X 
variables  includes  dummy  regional  variables  (Vietnam  is  divided  into  geographic 
regions),  urban/rural,  age  and  occupation  of  household  heads,  and  other  variables  of 
household composition and education.
6  
To estimate MTE or the impact distribution of the program by matching, we can 
implement two steps as follows. In the first step, we estimate impact for every household 
by a matching method. In this example, we perform matching based on the propensity 
score which is defined as the probability of receiving foreign remittances (regardless of 
remittance amount). The propensity scores can be estimated by logit or probit models. 
Then for each household who receives remittances, we calculate the difference in her 
observed outcome and the weighted average outcome of her matched households.  
                                                       
6 Readers can contact the author for detailed description of variables and estimation results.    7 
In the second step, we can use parametric regressions to estimate MTE. The main 
problem is to specify a functional form of  d) f(X, . In this example, to compare this semi-
parametric method with OLS regression we suppose that  d) f(X, has a form similar to 
(15), i.e.: 
2 ) , ( d d d X f θ γ + = ,                      (16) 
Hence: 
d MTE θ γ 2 + =                         (17) 
We can estimate  γ and  θ  by running OLS regression of estimated impact on  d  and 
2 d for remittance-receiving households.  
Table 1 presents estimates of γ and θ using OLS regression and the so-called semi-
parametric approach in which the propensity score matching is used to estimate program 
impact  for  remittance-receiving  households  in  the  first  step.  It  shows  that  the  two 
methods  give  rather  close  estimates.  However,  the  semi-parametric  method  produces 
higher standard errors compared with the OLS regression.    
Table 1: Impact of foreign remittances on per capita expenditure 
Dependent (outcome) variables 
Log of per capita expenditure  Per capita expenditure 
Estimators  Explanatory variables 
Mean  Std. Err.   Mean  Std. Err.  
Remittances   0.000018  0.000001  0.138280  0.016050  OLS regression 
Remittances squared 
  -7.15E-11  6.32E-12  -4.54E-07  1.37E-07 
Remittances   0.000016  0.000003  0.126540  0.029646  Matching with 1 nearest 
neighbor in the first step  Remittances squared 
  -6.54E-11  3.02E-11  -4.46E-07  3.21E-07 
Remittances   0.000013  0.000002  0.117577  0.026430  Matching with 3 nearest 
neighbors in the first 
step  Remittances squared 
  -5.09E-11  2.25E-11  -3.90E-07  2.91E-07 
Remittances   0.000011  0.000002  0.129772  0.022332  Kernel matching with 
bandwidth of 0.05 in the 
first step  Remittances squared 
  -4.26E-11  1.61E-11  -4.55E-07  3.07E-07 
 
Note:  Both per capita expenditure and remittances are measured in thousand VND. 
“E-x” means “multiplied by 10
-x”, e.g. 
11 10
7.15 -
11 - 7.15E - =  
  The number of observations is 9188. Ratio of households receiving foreign remittance is 6.1%. 
           Standard errors for matching estimates are calculated using bootstrap with 500 replications.  
 
Source: Author’s estimation 
 
We  can  use  nonparametric  regression  of  estimated  impact  on  the  program 
variables. For example, Figure 1 graphs estimated impacts of remittance on per capita 
expenditure.     8 
 
Figure 1: Median spline curve of estimated impact of foreign remittances on per capita 
expenditure 
0
5
0
0
0
1
0
0
0
0
1
5
0
0
0
2
0
0
0
0
I
m
p
a
c
t
 
o
n
 
p
c
.
 
e
x
p
e
n
d
i
t
u
r
e
 
(
V
N
D
 
t
h
o
u
s
a
n
d
s
)
0 50000 100000 150000 200000 250000
Foreign remittances (VND thousands)
 
Source: Author’s estimation 
 
Finally, it should be noted that the matching method in the first step can allow for 
the estimation of the treatment effect at any interval of the treatment  as long as the 
number of the treatment observations in that interval is large enough to get the reliable 
estimation. 
 
5. Conclusions 
   
In fact, a program can be continuous. People can receive different levels of treatment 
from a program. In this case, program impact can be identified under the conditional 
independence assumption and estimated by the matching methods. In addition to ATT 
and ATE, MTE can be defined, which measures the additional impact that people can 
gain when receiving an additional program level. These parameters can be estimated by a 
semi-parametric  approach  with  two  steps.  In  the  first  step,  the  treatment  effect  at 
different values of the conditioning and program variables is estimated by the matching 
method.  In  the  second  step,  the  distribution  of  program  impact  and  MTE  can  be 
estimated non-parametrically or parametrically using the program impact estimates from 
the  first  step.  Compared  with  OLS  regression,  it  is  more  robust  in  the  sense  that  it   9 
requires a functional form of program impact, while OLS regression requires functional 
forms of both outcome and program impact.  
 
 
 
References 
 
Cochran, W. G. and S. P. Chambers, 1965, The Planning of Observational Studies of 
Human Population, Journal of the Royal Statistical Society, Series A (General) 128, No. 
2, 234-266. 
Cochran,  W.  G.,  1968,  The  effectiveness  of  Adjustment  by  Subclassification  in 
Removing Bias in Observational Studies, Biometrics, 24, 295-313. 
Rosenbaum,  P.  and  Rubin  R.,  1983,  The  Central  Role  of  the  Propensity  Score  in 
Observational Studies for Causal Effects, Biometrika, 70 (1), 41-55. 
Rubin,  D.,  1974,  Estimating  Causal  Effects  of  Treatments  in  Randomized  and  Non-
Randomized Studies, Journal of Educational Psychology, 66, 688-701. 
Rubin, D., 1977, Assignment to a Treatment Group on the Basis of a Covariate, Journal 
of Educational Statistics, 2 (1), 1-26.   
Rubin, D., 1979, Using Multivariate Sampling and Regression Adjustment to Control 
Bias in Observational Studies, Journal of the American Statistical Association, 74, 318–
328. 
Rubin, D., 1980, Bias Reduction Using Mahalanobis-Metric Matching, Biometrics, 36 
(2), 293–298.   
 
 
 
 
 
 

Assignment to a Treatment Group on the Basis of a Covariate,

Bias Reduction Using Mahalanobis-Metric Matching,

Estimating Causal Effects of Treatments in Randomized and NonRandomized Studies,

The effectiveness of Adjustment by Subclassification in

The Planning of Observational Studies of Human Population,

Using Multivariate Sampling and Regression Adjustment to Control Bias in Observational Studies,

Estimating Impact of a Continuous Program under a Conditional Independence Assumption

Estimating Impact of a Continuous Program under a Conditional Independence Assumption

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics