Kettunen, Aija

Schuring, Merel

Julkari

                 DATA BRIEF  30  ●   DECEMBER  2016 Strategies to investigate the cost-effectiveness of  employment measures in a non-experimental setting                                            Key issues    Cost-effectiveness evalua-tion is a way to inform deci-sion makers to get best val-ue for money, but evidence on cost-effective re-employment policies relat-ed to health is limited.  Evaluation needs data on outcomes (e.g. employ-ment, health, wellbeing, social inclusion) and the costs of achieving them.  Observational studies with appropriate analytical techniques can produce cost-effectiveness infor-mation rapidly and with reasonable resources.  To use the new research strategies, more data should be collected sys-tematically. Especially on outcomes related to health and wellbeing, and robust effectiveness studies need to be conducted.  BACKGROUND: INVESTIGATING COST-EFFECTIVENESS This policy brief describes evaluation methods which may help decision mak-ers to develop cost-effective policies to reintegrate people with health prob-lems and long term unemployed into labour market. The focus is on non-experimental study settings and pre-conditions to conduct cost-effectiveness studies. Cost-effectiveness evaluation, also called an economic evaluation, offers in-formation to policy makers to use the scarce resources so that they can achieve the best possible performance and to get best value for money. The key questions are 1) is the policy effective in generating aspired outcomes, and if ‘yes’ 2) are the outcomes worth paying the costs of resources used compared to other ways to use the resources1, 2. That means that investigat-ing cost-effectiveness requires comparing the resources (the costs) used by each policy to generate outcomes with the outcomes achieved (the effec-tiveness). Quite many of the evaluations of active labour market policies (ALMP) ignore the ‘cost’ side3. Only few evaluation studies have recognised the role of health in labour market and in ALMPs, either. When a policy has both lower costs and better effectiveness compared to other policies, it is a cost-effective policy. However, when a policy is more effective (better outcomes), but costs more, policy makers have to decide whether the better outcomes are worth paying. Cost-effective or ‘worth pay-ing’ does not mean that the policy needs to bring savings. If the aim of the policy is to improve health, wellbeing and quality of life, it is ‘worth paying’ when the outcome gains are greater than achieved by spending the same resources to other policies aiming the same outcomes.   Judging the ‘worth’ of outcomes would be easier if the outcomes were meas-ured in monetary values. This is the case in one type of cost-effectiveness evaluation, cost-benefit analysis. Although it is possible to value employment outcomes into money, it is challenging to put monetary values to health and wellbeing outcomes.  A randomised controlled trial (RCT) is the best design to provide robust evi-dence on the cost-effectiveness of policy measures4. It is resource and time consuming, sometimes felt unethical, and it is quite often not feasible e.g. in evaluating small or local employment programmes. Quasi-experimental de-signs are more often feasible instead of RCT, but do not give fast answers (example5). New instruments need to be developed to make comparisons possible. Observational designs have also been used: costs and outcomes are observed in ‘non-experimental’ settings, and statistical methods are used to adjust for differences between the affected individuals and the comparison group (example6). Modelling is an alternative to use previous evidence to study local settings and when politicians need results rapidly (example7). Be-cause models require data from previous evaluations, observational studies or routine management information systems, reliability of models is depend-ent on the quality of the data available.   Authors:  Aija Kettunen, Dr.Pol.Sc. Diaconia University of Applied Sciences Merel Schuring, PhD Erasmus University Medical Center      Kettunen and Schuring Strategies to investigate the cost-effectiveness of employment measures in a non-experimental setting           THL – Data brief  30/2016            2                                  COST-EFFECTIVENESS OF EMPLOYMENT PROGRAMMES FOR PERSONS WITH MENTAL HEALTH PROBLEMS: STANDARDIZED RE-EMPLOYMENT RATIO INSTEAD OF EXPERIMENTAL DESIGN Commissioned by the Dutch Ministry of Social Affairs, a new instrument was developed to evaluate the effectiveness of employment programmes for un-employed persons with health problems in a non-experimental setting8,9.  The standardized re-employment ratio (SRR) gives insight in the added value of a specific measure compared to ‘care as usual’. The observed number of per-sons entering paid employment is divided by the expected number of persons entering paid employment. SRR higher than 1 indicates that the employment strategy results in more persons in employment than expected. Two longitudinal cohort studies were used to develop a prognostic model to estimate the expected number of persons entering paid employment: 1) a random sample of Dutch unemployed persons from Statistics Netherlands with information on health, personal and household characteristics and employ-ment transitions during 10 years of follow-up.8 2) A cohort of long-term unem-ployed persons with psychological health problems living in the four largest cities with information of perceived health, personal and household character-istics and employment status two years follow-up10. Factors that were most important in predicting entering paid employment were combined to estimate the probability of entering paid employment for each individual. The developed instrument was applied in the Netherlands to evaluate the ef-fectiveness of three different employment programmes: programme A (IPS for young patients with a first episode of psychosis), programme B (IPS for chronic psychiatric patients), and programme C (integrated employment programme for unemployed persons with psychological problems). For a period of one year, each person that entered paid employment was registered to calculate the observed number of persons. The expected number of persons was esti-mated with the developed prognostic model. Programme A had an SRR value of 1.9. This means that this specific employment measure resulted in a 90% increase of the number of re-employed persons compared to ‘care as usual’. Programme C had an SRR value below 1. This means that the programme did not have an added value compared to ‘care as usual’ in the first year. However, the one year period of registration of the persons may have been too short, because of the ‘first train, then place’ strategy of this specific employment programme. A longer follow-up period was recommended. Table 1.  Standardized re-employment ratio (SRR) and return on in-vestment period (ROI) of three employment programmes for unemployed persons with mental health problems   Probability of entering paid employment   Observed (%) Expected (%) SRR ROI (years) Programme A (n=120) 29.3% 15,5% 1,89 6,2 Programme B(n=118) 16,5% 13,7% 1,20 >8 Programme C (n=340) 6,3% 9,5% 0,66 >8        SRR=standardized re-employment ratio, ROI=return on investment period Project: Cost-effectiveness of policies aimed at prolonging working careers - the role of health (EU-HEMP)  EU-HEMP is an EU-funded project studying the impact of health in increasing employment rate and labour market participation. This series of policy briefs is produced as part of the project.  The principal objectives of the project are the following:  1) Producing and gathering evi-dence on the cost-effectiveness of strategies aimed at reintegration of disabled people, people on sickness leave and long-term unemployed people into the la-bour market and policies aimed at prolonging careers of near-retirement people. In short, we study the role of health in increas-ing employment.  2) Policy recommendations for the design of more cost-effective social policies will be made based on data analyses, comparative policy analysis and review of best practices.  3) Creation of an international expert network in the field of rehabilitation, reintegration, disa-bility and health policies in con-nection with labour markets and focusing on the cost-effectiveness of these policies.   Website   The project is funded by the Euro-pean Commission (agreement no. VS/2014/0174) together with the National Institute for Health and Welfare.   Kettunen and Schuring Strategies to investigate the cost-effectiveness of employment measures in a non-experimental setting           THL – Data brief  30/2016            3                                In addition, a cost-benefit analysis was done of the three employment pro-grammes. An increase of persons entering paid employment resulted in saving costs for social benefits and medical consumption, and generating more in-come out of work. The return on investment period (ROI) was calculated, tak-ing into account the costs of the employment programmes. The ROI was 6.2 years for VIP Amsterdam and more than eight years for the other employment programmes. SIMULATION MODELLING: FAST BUT ONLY AS GOOD AS THE DATA One way to provide estimates of cost-effectiveness and pay-offs relatively rap-idly is to combine results from previous studies and data from different sources to simulate impact pathways (decision tree pathways). Simple decision analytic modelling simulates, what would happen to individuals, and what would be the costs, outcomes and economic consequences of different inter-ventions. It works also for small and local interventions, if there is (robust) evidence and data available. Simulation modelling has been used in UK e.g. on mental health promotion and mental illness7. The Centres for Economic Development, Transport and Environment of North Karelia granted funding from European Social Fund to study economic conse-quences of two ALMP measures, the highest-level increased pay subsidy and rehabilitative work activity in Eastern Finland11. These measures are intended for the job-seekers, who need a great deal of support and guidance with em-ployment, as well as with health, wellbeing, social participation and coping with life. A person’s wellbeing should be improved before re-employment is possible, but these outcomes of ALMP measures are not systematically col-lected and registered. This study wanted to recognise also the wellbeing out-comes and related consequences.  It was not possible to organise even quasi-experimental design, and therefore simple decision modelling was carried out. Four impact pathways were identi-fied: a) employment, when unemployment benefits are replaced by wages, b) the clarification of the plans e.g. education followed by employment, c) in-crease in wellbeing followed by reduction in the need for social and health services, and d) improved coping with life, civil participation and active citizen-ship followed by benefits to the community. Data on costs and outcomes were obtained from previous research, secondary data, statistics, and expert opinions. In particular, the Finnish data of social wellbeing and inclusion outcomes were limited. The employment pathway was the only one possible to fully populate with acceptable data and complete the calculations. The impacts on the distribution of income transfer costs were calculated separately. Our data enabled the estimation of some savings in mental health care utiliza-tion, which were small compared to, for example, the productivity gains. Even without proper information of the savings due to increased wellbeing, and when the productivity gain was taken into account, the pay-offs of the highest-level increased pay subsidy exceeded the costs. Costs of rehabilitative work activity exceeded the pay-offs.  Previous research indicates that ALMP measures may impact also individuals’ wellbeing in a positive manner, but existing studies on wellbeing outcomes are primarily qualitative or insufficient. In order to use modelling, we need (ro-bust) evidence and quantitative data about changes in the wellbeing of the participants. This means that instruments to systematically measure and gath-er information about the wellbeing outcomes need to be developed.  BIBLIOGRAPHY   1) Drummond, M. F.; Sculpher, M. J.; Torrance, G. W.; O'Brien, B. J. & Stoddart, G. L. (2005) Methods for the Economic Evaluation of Health Care Programmes. Oxford Univer-sity press.  2) Knapp, M. (2013) Making an economic case for better mental health services. In Thornicroft, G., Ruggeri, M. & Goldberg, D. (edit-ed). Improving mental health care: The global challenge, First Edition. John Wiley & Sons, Ltd.  3) Card, D., Kluve, J. & Weber, A. (2010) Active labour market policy evaluations: A meta-analysis, Economic Journal, 120, F452-477.  4) Knapp, M., Patel, A., Curran, C., Latimer, E., Catty, J., Becker, T., Drake, R. E., Fioritti, A. Kilian, R., Lauber, C., Rössler, W., Tomov, T., Van Busschbach, J., Comas-Herrera, A. White, S., Wiersma, D. & Burns, T. (2013) Supported employment: cost-effectiveness across six European sites World Psychiatry, 12 (1). 60-68.  5) Sarvimäki, M. & Hämäläinen, K. (2016) Integrating Immigrants: The Impact of Restructuring Active Labor Market Policies Journal of Labor Economics, 34(2): 479–508.  6) Jespersen, S. T., Jakob R. M. & Lars S. (2008) Costs and Benefits of Danish Active Labour Market Programmes.” Labour Economics 15 (2008): 859-884.  7) Knapp, M., McDaid, D. & Par-sonage, M. (eds.) (2011) Mental health promotion and mental illness prevention: the economic case 15972. Department of Health, London, UK.  8) Schuring, M., Robroek, S. J. W., Otten Ferdy, W. J., Arts, C. H. & Burdorf, A. (2013) The effect of ill health and socioeconomic status on labour force exit and re-employment: a prospective study with ten years of follow-up in the Netherlands. Scand J Work Environ Health. Vol. 39(2): 134-143.      Kettunen and Schuring Strategies to investigate the cost-effectiveness of employment measures in a non-experimental setting           THL – Data brief  30/2016            4                             BIBLIOGRAPHY   9) Schuring, M., Reeuwijk, K., & Burdorf, A. (2016) Evaluation of re-employment programmes for long term unemployed persons with health problems (Het monitoren van arbeidstoeleiding van mensen met grote afstand tot de ar-beidsmarkt en gezond-heidsproblemen) Department of Public Health, Erasmus University Medical Center, Rotterdam, the Netherlands  10) Schuring, M., van Rijn, R., Burdorf, A. (2016) Evaluation Fit4Work - effectiveness of a first place then train programme among unemployed persons with psychological problems (Evaluatie Fit4Work – effectiviteit van een integrale aanpak gericht op snelle toeleiding naar betaald werk van mensen met psychische problematiek) Department of Public Health, Erasmus University Medical Center, Rotterdam, the Netherlands.  11) Pehkonen-Elmi, T., Kettunen, A., Surakka, A. & Piirainen, K. (2015) Economic analysis of active measures for those who are diffi-cult to employ – Cases: highest-level increased pay subsidy and rehabilitative work activity. Publi-cations of Diaconia University of Applied Sciences A Studies 43. Helsinki: Diaconia University of Applied Sciences. Helsinki.                  National Institute for Health and Welfare Pl 30 (Mannerheimintie 166) 00271 Helsinki Puhelin: 029 524 6000  ISBN 978-952-302-796-1 (on-line) ISSN 2323-5179  http://urn.fi/URN:ISBN:978-952-302-796-1  www.thl.fi RECOMMENDATIONS  European Union Member States need cost-effective strategies to re-integrate long-term unemployed and people with health problems into labour market. Evidence on cost-effective policies is limited. Especially the role of health and wellbeing and data on costs is often missing to make conclusions of cost-effectiveness. Robust cost-effectiveness evaluations using experimental settings and large sample of people take time and need resources and are not always feasible. Ways to provide more rapidly and less costly conclusions on cost-effectiveness of re-employment strategies related to health should be developed. It is possi-ble, for example, to develop instruments to conduct cost-effectiveness evalua-tion even when the comparison group is not available. Models are available to simulate impact pathways (costs, outcomes and economic consequences) and make conclusions about cost-effectiveness. Conclusions of cost-effectiveness and cost consequences are possible by adding economic dimension to out-come studies. Adaption of results from one country to another country might be possible as well.  For all above mentioned strategies, better data on effectiveness of policy measures, especially on health and wellbeing outcomes, as well as data on costs to generate outcomes, is essential. The costs used by stakeholders other than those directly operating the ALMP measure should be considered as well: for example health and social care costs, costs for individuals and families as well as productivity costs. Data on savings and gains, e.g. productivity gains are as important as costs. Subsidies and other transfer payments are relevant in employment and health policies. For developing instruments to help evaluat-ing small and local programmes information on determinants of individuals entering payed employment is important. Also modelling requires this infor-mation as well as evidence on associations between health and wellbeing and service use. Collecting better routine data on employment, re-employment, health and wellbeing as well as service use is crucial for developing ways to provide evi-dence on cost-effectiveness, and should be developed. Better use of cost-effectiveness evidence in policy making requires greater understanding between policymakers, other stakeholders and researches on cost-effectiveness and economic evidence. Discussions between policy makers from social, health and labour market sectors and researches is important to develop understanding as well as improve using evidence and designing more cost-effective policies to re-integrate people with health problems into labour market.    

Strategies to investigate the cost-effectiveness of employment measures in a non-experimental setting

http://www.julkari.fi/bitstream/handle/10024/131622/URN_ISBN_978-952-302-796-1.pdf?sequence=1&isAllowed=y

Strategies to investigate the cost-effectiveness of employment measures in a non-experimental setting

Abstract

Similar works

Full text

Available Versions

Julkari