The study estimates the employment effect of vocational training programs for the unemployed in urban Russia. The results of propensity score matching indicate that training programs had a non-negative overall effect on the program participants relative to non-participants.Unemployment, transition economies, active labour market programs, evaluation, propensity score

Anton Nivorozhkin

Eugene Nivorozhkin

English

Research Papers in Economics

Upjohn Institute Working Papers Upjohn Research home page
2005
Do Government Sponsored Vocational Training
Programs Help the Unemployed Find Jobs?
Evidence from Russia
Anton Nivorozhkin
Göteborg University
Eugenity Nivorozhkin
University of Groningen
Upjohn Institute Working Paper No. 05-115
This title is brought to you by the Upjohn Institute. For more information, please contactir@upjohn.org.
Published Version
Article in Applied Economics Letters, 14(1-3) (January-February 2007): 5-10
Citation
Nivorozhkin, Anton, and Eugenity Nivorozhkin. 2005. "Do Government Sponsored Vocational Training Programs Help the
Unemployed Find Jobs? Evidence from Russia." Upjohn Institute Working Paper No. 05-115. Kalamazoo, MI: W.E. Upjohn Institute
for Employment Research.
http://research.upjohn.org/up_workingpapers/115Do Government Sponsored Vocational Training Programs Help the 
Unemployed Find Jobs?  
Evidence from Russia 
Anton Nivorozhkin
*, a and Eugene Nivorozhkin
*, b 
a Department of Economics, Göteborg University
** 
b Department of Financial Management, University of Groningen 
Upjohn Institute Working Paper no. 05-115 
(Accepted for publication in Applied Economics Letters) 
 
Abstract  
The study estimates the employment effect of vocational training programs for the 
unemployed in urban Russia. The results of propensity score matching indicate that 
training programs had a non-negative overall effect on the program participants 
relative to non-participants.  
 
Keywords: Unemployment, transition economies, active labour market programs, 
evaluation, propensity score. 
 
JEL classification: J24, J64, J68, C14. 
                                                 
* The authors would like to thank Michael Lechner, Ludmila Nivorozhkina, Lennart Hjalmarsson and 
Rick Wicks for providing useful comments, and Oksana Fedosova for research assistance. The support 
of the Institute of Independent Social Policy (Grant No. SP-02-2-12, Ford Foundation) and The W.E. 
Upjohn Institute for Employment Research (Grant No. 03-83-07) are gratefully acknowledged. The 
usual disclaimer applies. 
 
** Corresponding author:  Department of Economics, Göteborg University, Box 640, SE 405 30, 
Göteborg, Sweden. e-mail: econaniv@economics.gu.se   2
 
 
 
Do Government Sponsored Vocational Training Programs Help the 
Unemployed Find Jobs?  
Evidence from Russia 
 
 
 
 
 
 
Abstract  
This study estimates the employment effect of vocational training programs for the 
unemployed in urban Russia. The results of propensity score matching indicate that 
training programs had a non-negative overall effect on the program participants 
relative to non-participants.  
 
 
Keywords: Unemployment, transition economies, active labour market programs, 
evaluation, propensity score. 
 
JEL classification: J24, J64, J68, C14. 
   3
1.  Introduction 
 
This note contributes to the scarce literature on the labour policies in transition 
economies by investigating the employment effect of active labour market programs 
(ALMP) in the Russian Federation. 
Internationally, the usefulness of ALMPs is often a subject of scepticism 
among politicians as well as employers and jobseekers themselves (e.g., Heckman et 
al., 1999; Grubb and Martin, 2001; Kluve and Schmidt, 2002). That and tight budget 
constraints suggest the need for regular monitoring and evaluation of labour market 
programs (O’Leary et al., 2001). Although evaluation of ALMPs became a common 
practice in many transition economies of Central and Eastern Europe, until recently 
there was no rigorous econometric evaluation of ALPMs in Russia (Gimpelson, 2002; 
World Bank, 2003). 
Our paper evaluates the employment effect of government sponsored 
vocational training programs. Due to unavailability of a countrywide database, we 
used administrative data from the Public Employment Office (PEO) of Rostov-on-
Don, the administrative centre of the Southern Federal District of Russia. The 
administrative data was combined with a follow-up survey data on sampled 
unemployed individuals to trace their work history after leaving the PEO.  
Using the propensity score matching we compare the employment 
probabilities of training programs participants with a control group of non-
participants. We also controlled for potential heterogeneity in the effectiveness of 
training programs for blue and white collar occupations.
1 
                                                 
1 Professions which were demanded by the local labor market and thus for which training was offered 
during the period under investigation included accountant, secretary, waitress, bartender, car 
mechanical, track and bus drivers, as well as some others.   4
Section 2 presents the methodology and the data. Section 3 presents the results 
and section 4 concludes the paper. 
 
2. The dataset and sample selection 
 
Registration with the PEO is a pre-requisite for participation in government-
sponsored training programs, so our primary data came from the unemployment 
registry maintained by the PEO of Rostov-on-Don.
2 To collect the data regarding 
individuals’ employment status after leaving the PEO, we implemented a follow-up 
house-to-house survey. The survey sample consisted of random sample of 2,000 
individuals registered with the PEO of Rostov-on-Don in the year 2000. The follow-
up survey was conducted in September 2002. The overall survey response rate was 
77.3%, about the same for both participants in training programs and non-
participants.
3 Our final sample included 1,547 individuals. Among 406 individuals 
who participate in programs, 152 underwent training for blue-collar professions and 
254 underwent training for white-collar professions. This study focused on the 
employment effects of vocational training programs, and therefore individuals 
participating in other types of ALMPs (e.g., public works or start-up grants) were not 
included in the sample. 
The outcome of interest was employment probability.  The follow-up survey 
questions were constructed to capture both short-term and long-term effects of the 
training programs. First, the survey respondents were asked whether they found a job 
                                                 
2 According to the law “On Employment of the Population in the Russian Federation,” an unemployed 
individual is one who simultaneously satisfies the: (1) belongs to the labor force; (2) is presently 
without a job and income; (3) is actively searching for a job; (4) is willing to take on a job; (5) has 
applied to a PEO for assistance in finding a job. 
3 The two main reasons for non-response were refusal to let the interviewer in and refusal to answer the 
questions. On some occasions it was impossible to locate an individual at the provided address.   5
after leaving the PEO. Second, they were asked whether they were employed twelve 
months after leaving the PEO.  
According to simple statistics, program participants were generally more 
likely to find a job upon leaving the PEO relative to non-participants. Among non-
participants only 85% found a job comparing to 94% of blue-collar training programs 
participants and 88% of white-collar programs participants. Twelve months after 
leaving the PEO, the proportion of employed individuals decreased in all groups. 
Only 80% of non-participants were employed, comparing to 82% of blue-collar 
training program participants and 79% of white-collar participants.  
 
3.  Empirical strategy and results 
 
To estimate the empirical model we employed the propensity score 
methodology. Rosenbaum and Rubin (1983, 1984) initiated the literature on matching 
methods. The authors proposed statistical matching on the basis of predicted 
probability of participation in the training program, i.e., propensity score. By 
matching one try to ex-post mimic randomization in control and treatment group in 
experimental studies. Intuitively, this means that if observations in control and 
treatment group are similar in all observed characteristics than participation in the 
training program may explain labour market outcome. 
Comparing to other econometric methods matching has two major advantages. 
First, it provides a convenient test of overlap of observed covariates between 
treatment and control group. Moreover, if sufficient overlap is achieved treatment 
effect is estimated non-parametrically.    6
In recent years matching received a lot of attention in economic literature, e.g. 
Heckman et al. (1997, 1998), Dehejia and Wahba (1998), Lechner (2002), Smith and 
Todd (2004). Authors emphasize that validity of matching depends crucially of the 
absence of unobserved effects. To meet this assumption we selected variables 
expressing the pre-unemployment work history of individuals and their social-
demographic and educational characteristics. We also included variables serving as 
proxies for individuals’ motivation towards employment and period of inflow into the 
unemployment registry. Table 1 reports mean values for the variables describing 
individuals in the training and control samples.  
To estimate propensity score we followed the algorithm suggested by Dehejia 
and Wahba (1998, 2002):
4 
 
1.  Start with a logit function with linear covariates to estimate the 
propensity score. 
2.  Rank all observations by the estimated propensity score (from lowest 
to highest). 
3.  Impose “common support” condition, i.e. discard control group 
observations with estimated propensity score less than the minimum, 
or greater than the maximum estimated propensity score for training 
group observations. 
4.  Split the sample in 5 blocks of equal score interval and test whether the 
average propensity scores of training and control observations are the 
same in every block.  
                                                 
4 The algorithm for estimation of propensity score and for computation of the average treatment effect 
on treated (ATT) uses the Stata  programs developed by O.Baker and Ichino (2002).   7
           7.  Test that the means of each covariate do not differ between the trainees 
and control in every block.  
 
Following the algorithm we estimate logit function to predict probability of 
participation in training program and test for balance of covariates. In almost all cases 
the means were equal at the 5% confidence level, and none of the covariates 
systematically failed the test in all the blocks. The final distribution of training and 
control observations is presented in Figures 1 and 2. 
To accurately compute the ATT one should precisely match the training and 
control groups on the basis of propensity score. In practice it is never possible to 
match the scores precisely, however, and thus four alternative matching methods were 
used and will be compared: stratification, nearest neighbourhood, radius, and kernel 
matching. A complete description of the matching estimators used in this paper may 
be found in O.Baker and Ichino (2002).  
The results of the estimation are presented in Table 2. To match treatment and 
control group we applied four different algorithms. Moreover, radius matching was 
applied twice with different specification of radius. According to results, individuals 
who were trained to become blue-collar workers were more likely to find employment 
comparing to untrained counterparts. Yet, no significant effect was detected in the 
long run. The long-run effect for the white-collar training is even negative although 
statistically insignificant. The estimations broadly agree with each other, i.e. positive 
for blue-collar trainees upon leaving the employment office, but not statistically 
significant in all other cases.  
 
   8
4. Discussion and conclusions 
 
The main conclusion we can draw from our evaluation results is that the 
vocational training programs conducted by the Public Employment Office of Rostov-
on-Don overall had a non-negative effect on the employment probabilities of program 
participants relative to non-participants. Participants in blue-collar training had a 
discernible immediate positive effect, while the participants of white-collar programs 
did not. These results must be viewed with cautions; the positive effect of blue-collar 
training may be explained by the larger number of blue-collar vacancies in the labour 
market of Rostov-on-Don, a big industrial city. For example, in 1999, employment in 
manufacturing increased by 9.3% while employment in services sector remained 
static. Moreover, some of the blue-collar training programs can potentially be targeted 
at the labour demand of a specific firm, although the PEO officers did not indicate to 
us the existence of any formal agreements to that effect.  
Can these results for one particular city be generalized to the rest of Russia? 
Indeed, there are reasons to believe in the existence of a substantial disparity in the 
development of different Russian regions. Nevertheless, we believe that the results 
here can be generalized to a lager group of industrial cities in Russia. According to 
Russian labour laws, the legislative framework determining eligibility for 
participation in training programs is uniform. Moreover, large cities in Russia tend to 
be similar to each other in having a diversified industrial structure and well-developed 
educational and training infrastructure. Finally, the system of population registration 
and the under-developed housing market discourage labour mobility, creating 
stagnant unemployment pools in the cities. Thus labour market processes in large 
industrial cities tend to be very similar.   9
From a policy standpoint, the results of our paper are modestly encouraging. 
The effects of training programs tend be rather limited in both advanced industrial 
economies and advanced transition economies. Considering the relatively low level of 
expenditure on ALMPs in Russia and the lack of PEO experience, the effectiveness of 
some of the training programs is rather surprising. The variation in program effects 
across different types of training stresses the importance of monitoring the efficient 
program mix and providing appropriate infrastructure for various types of skill 
enhancing programs for unemployed individuals. 
 
 
References 
 
Dehejia, R., Wahba, S. 1999. “Causal effects in non-experimental studies: 
reevaluating the evaluation of training programs.” Journal of the American 
Statistical Association, 94, 1053-1062. 
Dehejia, R., Wahba, S. 2002. “Propensity score matching methods for 
nonexperimental causal studies.” Review of Economics and Statistics, 84, 151-
161. 
Gimpelson, V. 2002. Comments on Lawrence Thompson's evaluation report “Russia: 
Bank assistance for social protection.” Operations Evaluation Department. 
Washington, DC: The World Bank. 
Grubb, D., and Martin, J. P. 2001. “What works and for whom? A review of OECD 
countries´ experiences with active labour market policies.” Swedish Economic 
Policy Review, 8, 9-56. 
Heckman, J., LaLonde, R., and Smith, J. 1999. “The Economics and Econometrics of 
Active Labor Market Programs.” In Handbook of Labor Economics, vol. 3A, 
North-Holland, pp.1865-2097.   10
Heckman, J., Ichimura, H., Todd P. 1997. “Matching as an econometric evaluation 
estimator: evidence from evaluating a job training program.” Review of 
Economic Studies, 64, 605-654. 
———. 1998 “Matching as an econometric evaluation estimator.” Review of 
Economic Studies, 65, 261-294.  
Holland, P. W. 1986. “Statistics and causal inference.” Journal of the American 
Statistical Association, 81, 945-960. 
Kluve, J., Schmidt, C. M. 2002. “Can training and employment subsidies combat 
European unemployment?” Economic Policy: A European Forum, 35, 409-
443. 
Lechner, M. 2002. “Program heterogeneity and propensity score matching: An 
application to the evaluation of active labor market policies.” Review of 
Economics and Statistics, 84, 205-220. 
O’Leary, C., Nesporova A., Samodorov, A. 2001. Manual on evaluation of labour 
market policies in transition economies.” Geneva: ILO. 
O.Backer, S., Ichino, A. 2002. “Estimation of average treatment effects based on 
propensity scores.” The Stata Journal, 2, 358-377. 
Rosenbaum, P., Rubin, D. 1983. “The central role of the propensity score in 
observational studies for causal effects.” Biometrika, 70, 41--55. 
———. 1984. “Reducing bias in observational studies using the subclassification on 
the propensity score.” Journal of the American Statistical Association, 84, 
1024-1032. 
Smith, J., Todd P. 2004. “Does matching overcome LaLonde’s critique of 
nonexperimental estimators?” Journal of Econometrics, 125, 305-353. 
World Bank. 2003. The Russian Labor Market: Moving from Crisis to Recovery. 
Washington, DC: World Bank. http:// www.worldbank.org.ru/ECA/ 
Russia.nsf/ECADocByLink/ 91E880C04CBEC533C3256E27004B1C11.   11
 
Appendix 
Table 1: Mean values for variables in training and control groups.
 
Variable 
Blue-collar 
training  
White-collar 
training  
Control  
Male  0.428 0.130 0.337 
Age≤20 0.224  0.165 0.159 
Age 21-30  0.408  0.398 0.296 
Age 31-40 
0.171 
0.291 0.180 
Age 41-50 
0.151 
0.126 0.254 
Age >51 
0.046 
0.019 0.111 
Married 
0.329 
0.402 0.449 
Number of children  0.454  0.500 0.365 
Disabled 
0.026 
0.031 0.058 
Disadvantaged 
0.111 
0.055 0.056 
University education 
0.191 
0.465 0.340 
Technical secondary education  0.177  0.280 0.270 
General secondary education  0.336  0.165 0.213 
Only primary education or less  0.296  0.091 0.177 
No work experience  0.388  0.295 0.249 
Work experience 0-5  0.184  0.185 0.155 
Work experience 6-15  0.224  0.27 0.203 
Work experience >15  0.204  0.244 0.393 
Unskilled worker  0.322  0.24 0.189 
Blue-collar worker  0.132  0.047 0.102 
Skilled blue-collar worker  0.269  0.126 0.220 
White-collar worker 
0.263 
0.465 0.388 
Skilled white-collar worker 
0.013 
0.114 0.102 
Pre-unemployment average monthly wage   241.15  475.05  559.15 
State  ownership  0.407 0.406 0.377 
Private  ownership  0.184 0.205 0.205 
Mixed  ownership  0.171 0.197 0.262 
No data on ownership  0.237  0.193  0.155 
Looking for permanent, full-time job  0.605 0.685 0.691 
Winter  0.243 0.146 0.219 
Spring  0.263 0.220 0.230 
Summer  0.217 0.354 0.266 
Fall  0.276 0.279 0.215   12
Figure 1: Distribution of estimated propensity score, blue-collar 
training and conrol groups
0
50
100
150
200
250
300
350
1234567
The region of common support is [0.01277477 - 
0.56281613]
N
u
m
b
e
r
 
o
f
 
o
b
s
e
r
v
a
t
i
o
n
s
Control
Training
 
 
 
 
 
Figure 2: Distribution of  estimated propensity score, white-collar 
training and control groups
0
50
100
150
200
250
300
350
400
450
12345
The region of common support is [0.00720224 - 
0.58339802]
N
u
m
b
e
r
 
o
f
 
o
b
s
e
r
v
a
t
i
o
n
s
Control
Training  13
Table 2: The effect of training on participant relative to non-participants 
  Effect of blue-collar training   Effect of white-collar training  
Matching method 
Effect after leaving the 
PEO 
Effect 1 year after 
leaving the PEO  
Number of 
observations 
Effect after leaving the 
PEO  
Effect 1 year after 
leaving the PEO 
Number of 
observations 
     Training Control      Training Control 
Stratification   0.094 
(3.74) 
0.005 
(0.14) 
152 1046 -0.000 
(-0.01) 
-0.024 
(0.90) 
254 1097 
Nearest 
neighbourhood 
0.122 
(3.02) 
0.021 
(0.42) 
152 165  -0.017 
(-0.58) 
-0.019 
(-0.46) 
254 237 
Radius (r=0.0001)  0.095 
(1.317) 
-0.046 
(-0.48) 
71 125 0.017 
(0.33) 
-0.000 
(-0.001) 
114 166 
Radius (r=0.0005)  0.084 
(2.253) 
-0.014 
(-0.27) 
118 358  -0.022 
(-0.60) 
-0.030 
(-0.66) 
203 408 
Kernel 
(bw=silverman) 
0.096  
(3.749) 
0.013 
(0.40) 
152 1046 0.002   
(0.07) 
-0.025 
(-0.85) 
254 1097 
t-statistics in parentheses. Standard errors were calculated by bootstrap method (200 replications)  14
 

Can training and employment subsidies combat European unemployment?” Economic Policy:

Causal effects in non-experimental studies: reevaluating the evaluation of training programs.”

Comments on Lawrence Thompson's evaluation report “Russia: Bank assistance for social protection.” Operations Evaluation Department.

Does matching overcome LaLonde’s critique of nonexperimental estimators?”

Estimation of average treatment effects based on propensity scores.”

Manual on evaluation of labour market policies in transition economies.”

Matching as an econometric evaluation estimator.”

Program heterogeneity and propensity score matching: An application to the evaluation of active labor market policies.”

Propensity score matching methods for nonexperimental causal studies.”

Statistics and causal inference.”

The central role of the propensity score in observational studies for causal effects.”

The Economics and Econometrics of Active Labor Market Programs.”

The Russian Labor Market: Moving from Crisis to Recovery.

What works and for whom? A review of OECD countries´ experiences with active labour market policies.”

Do Government Sponsored Vocational Training Programs Help the Unemployed Find Jobs? Evidence from Russia

http://research.upjohn.org/cgi/viewcontent.cgi?article=1132&context=up_workingpapers

Do Government Sponsored Vocational Training Programs Help the Unemployed Find Jobs? Evidence from Russia

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics