Wednesday, 29 February 2012

Work experience: we still need better evidence

Last week I wrote here that the debate on work experience seemed almost entirely divorced from the evidence of whether work experience actually improved the employment opportunities of jobless young people.  I pointed out that, when Iain Duncan Smith and Chris Grayling argue that around half of those on the scheme leave benefits within 13 weeks, this in itself tells us nothing about the success of the programme, since many would have left benefits without the scheme.  It is worth reproducing again Inclusion's chart here, since it remains the best data we have: 

What has happened since then, and have we learned more?  Unfortunately, exaggerated claims appear to continue.  The Prime Minister, at last week's PMQs, stated
"around half of them [young people on the scheme] are actually getting work at the end of these schemes." 
On the basis of the data and information we have at the moment, this is clearly false, as pointed out here. While the Prime Minister was presumably badly briefed, there was much less excuse for Sarah Teather two days later, who repeated again (on Any Questions):
"50% of people who start these schemes have got a job"
Now, the fact that Ministers have made exaggerated (and sometimes false) claims for the success of the scheme definitely does not imply it doesn't work.   For that reason, I argued 
"DWP should attempt to produce a proper counterfactual analysis that would allow us to come to a considered judgement on the programme's success."  
This would involve  the construction of a proper "control group" of non-participants, similar to participants on the scheme, so we could see whether outcomes were better for the latter.  And interestingly, Chris Grayling, on the Today programme, hinted that DWP were doing just that:  
"Now we’re now crunching further numbers and all of the evidence we can see is that this does better than simply leaving people on JSA. It actually helps more young people get into work."
This is very encouraging.  DWP should never have published the misleading analysis here, which does not even mention the need for a counterfactual analysis to assess the success of the programme. 

But - and here I would like to be resolutely optimistic, both about evaluation and analysis, and the programme itself - there is an opportunity here to improve the quality of the current debate.   DWP should publish the evidence referred to by Chris Grayling as soon as possible, and open up the data and analysis to outside scrutiny. This would help explain both what welfare-to-work programme can and cannot do, and how they should be evaluated.   My view - shared by most of those who study and evaluate such programmes - is that work experience is likely to improve outcomes for young people, albeit not as much as implied by Ministers' claims.  I hope I am right, and look forward to seeing the evidence. 

Furthermore, the current intense scrutiny of the programme should be taken as an opportunity, not to rubbish it, but to improve it.  The ACEVO Commission on Youth Unemployment (of which I was a member) found:
"currently work experience placements are too often short, of poor quality, with young people given little to do and the placement poorly linked to their wider education or the advice and guidance they receive"
This is to a large extent the responsibility of employers at least as much as government. Channel 4's Factcheck found that only one in five of those doing work experience at Tesco was offered a permanent job - and unfortunately this is entirely consistent with what the ACEVO Commission was told.  

While this does not contradict the analysis above (they might have got jobs somewhere else) this suggests to me that Tesco are fully abiding by the spirit of the scheme, which is supposed to offer the chance of a permanent job to those who successfully complete a placement.  Research by Alex Bryson, now at NIESR, and others found back in the 1990s that this was the key to success for such schemes.  Do we really think that only one in five of the young unemployed people who go on the current scheme proves to be sufficiently competent and motivated to fill a permanent position?  To make the scheme work for everyone - employers, government, and most of all young unemployed people - the most important point is that it needs to offer them a fair chance of a proper job at the end. 


  1. The success of work experience placements (if we take success to mean the participant moving into paid work) is heavily reliant on matching the candidate with a relevant role for their educational background, alongside careful structuring of the placement by the host organisation.

    Admittedly this is circumstantial evidence, but there appears to be a propensity among (some) Job Centre Plus advisors to throw people into irrelevant placements in which there is a high likelihood the candidate will not stay beyond their 8 week work experience period.

    Rather than focusing on major national employers, there should be greater emphasis from DWP on engaging local employers who can provide tailored, relevant schemes for young job seekers. This is the approach we have fostered at Inspiring Interns (a London-based graduate internship agency), and it has lead to a placement to job conversion rate of around 65%.

  2. My question is... that even if someone on benefits who goes for a work experience scheme ends up getting hired for an open position, then that means that someone else who would have applied and gotten for that open position now finds themselves unemployed. So it hardly helps the overall unemployment levels, but it does give a chance to the long term unemployed with no track record to "prove themselves" in a trial before securing a permanent position.

    What would be better is the creation of new roles that is beneficial to the nation, that doesn't "steal" vacancies for jobseekers. This could be working with charities for example, who without the offer of the government covering benefits for a worker, would have been unable to otherwise afford them. While the worker won't be getting a job at the end of the stint, at least they have built on their CV, their skills, their confidence, and crucially a letter of recommendation from the charity or DWP about their job performance.

    This charity route will also benefit the community, and change the thinking around benefits that it is not free, and that people on benefits still need to contribute to society to get them. Charities are also less inclined to "abuse" the free labour, and will genuinely be grateful for the extra help.

  3. Boon is right. Even if Work Experience is a success on Jonathan Portes’s criteria, it still does not mean WE is on balance beneficial, because the jobs found by former WE employees may just be jobs stolen from others. (By “Jonathan Portes’s criteria”, I mean the criteria he set out in Jonathan's last sentence above.)

    There is however a reason in theory for thinking that artificially boosting the employment chances of youths will result in an aggregate increase in employment (it’s a reason cited in “Creating Jobs” published by the Brookings Institution, 1987, ed. J.L.Palmer)

    Just as there is a NAIRU for the workforce as a whole, there is a NAIRU for specific types of labour: that is, there is a level of unemployment amongst specific types of labour at which wage increases for that type of labour start to contribute to inflation in a serious way.

    The relatively high levels of unemployment amongst youths means that youth unemployment is probably well above “youth NAIRU”. Ergo any sort of artificial boost for youth employment (WE type schemes or a subsidy of youth employment) will raise employment amongst youths without exacerbating inflation. Ergo such jobs will tend to be NET ADDITIONS to aggregate employment.

    There is another reason for thinking that WE type jobs will at least to some extent be NET ADDITIONS to numbers employed, even if such schemes do not concentrate on specific types of labour, like youths. I set out these reasons here:

    However this is all very abstract and theoretical stuff: ten miles above the heads of 99.99% of those who sound of on this subject. I am 99.99% confident that the human race will never understand labour markets.

  4. Just wish to point out the continued objection to the graph on the basis that it does not have a common axis.

    As we do not know at what point in their JSA the applicants joined the WE scheme it is never going to be possible to be able to compare the two in timescale success.

    All we know is that they have been unemployed for at least 3 months, which is when almost 60% of all JSA claimants are already off JSA, and higher for the demographic of interest and up to 12 months, when almost all of the relevent demographic are off JSA.

    Perhaps the 50% after three months on the WE is good when you factor in these figures, perhaps.

    What I DO know is that this graph is flying around the internet as 'Proof' that people are more likely to get a job without WE then with it, adding more heat but no light at all.