The Impact of Careers Guidance for Employed Adults in Continuing Education

Abstract

The validity of the matching estimator in programme evaluation depends on the completeness of the set of variables used for matching. When an attitudinal variable is relevant for the participation decision, but is either unmeasured or measured only after entry to the programme, estimates of effects may be biased or hard to interpret. This issue is investigated with data from an evaluation study of careers guidance for employed adults, which utilised the method of propensity score matching. Job satisfaction, measured shortly after entry to the programme, was found to be strongly associated with participa-tion, but may itself have been influenced by the early experience of careers guidance. Estimates of the impacts of guidance on several post-programme education and training outcomes are considered, both including and exclud-ing the job satisfaction measure from the participation model. Data experiments with adjusted values of job satisfaction are also performed. It is found that estimates of treatment effects are highly sensitive to these variants, and respond in a non-monotonic fashion. The implications for evaluation methodology are discussed.