Statistics – Misuse

My main concern about the Government’s approach to statistics on ESA is that they are often deliberately misused to the disadvantage of people with genuine disabilities. For example a story from the Sunday Telegraph from 30 March 2013 used the headline ‘900,000 choose to come off sickness benefit ahead of tests’ and the copy was punctuated with quotes from Tory Chairman Grant Shapps. The implication was that people who had previously been on Incapacity Benefit and had been migrated across to ESA weren’t really ill or disabled in the first place, and were, in some way, playing the system.

This simply isn’t the case. Of the 600,000 people who have been migrated from Incapacity Benefit over the past two years, only 19,700 had dropped their claim. The figure of 900,000 refers to all those who have made new claims for ESA since its introduction over four years ago, but who have since withdrawn their application before undergoing a face to face assessment. These people weren’t claiming the benefit before and generally drop out the system for perfectly innocent reasons – often they become ill, apply as a precaution, but withdraw when they get better.

The far less significant figure of 19,700 is the one that should have featured in the headline. Once I realised this I complained to the Press Complaints Commission, and the paper have subsequently published a letter from me and noted my concerns on their website. I wrote about this story for Total Politics here and here, and for Third Force News here.

Another example relevant to ESA comes from a DWP press release from October 2010, which attempted to suggest that three quarters of applicants were in fact fit for work. However when you delve deeper you realise that this figure includes the 39% of applicants who withdraw their claim before going through with their assessment and doesn’t take account of appeals.

In April 2013 we had Iain Duncan Smith in the Mail and the Express referring to one million workshy benefit claimants, when in reality, one third of these are people in the WRAG, a group who have been certified as medically unable to work for the time being (with another third single parents looking after children of school age).

This steam of stories has prompted me to call for a Work and Pensions Select Committee inquiry into this issue. We did raise these issues in a one-off evidence session with Iain Duncan Smith on 9 December 2013. I wrote up some reflections on how this went for LabourList on 14 December.

I was disappointed that the BBC ran a similar story in January 2014 that said ‘more than a million [ESA applicants] withdrew their claims after interviews’. In addition to the my concern about the implication claimants were playing the system, it is also incorrect to suggest that the statistics discriminate between people that withdraw their applications before a face-to-face assessment and after – they simply refer to people withdrawing their application before they receive a final decision. I notice that subsequent revisions of this story have removed this reference, and that the DWP press release that prompted the story says instead ‘More than a million others withdrew their claims before reaching a face-to-face assessment’. However this appeared two days after the story on the BBC, leaving open the possibility that a draft was circulated in advance with this inaccurate quote included.

DWP Statistics Policies
Possibly in response to criticism of Ministers’s use of ESA statistics, on 25 March 2014 DWP published a series of documents on their website on how they produce the main WCA outcomes statistics:

  • The methodology document explains how statisticians draw together three data sets from DWP, Atos and HMCTS, with the process initiated when the quarterly appeals data arrives. The complexity of the process is relevant to Ministers’ refusal to publish reconsideration statistics. On page 4 it names the IT supplier they use; on pages 5 and 7 it explains the time it takes to publish the statistics – something I discuss here; on page 6 it explains that DWP’s database doesn’t include Fit for Work decisions from Decision Makers – it just contains Atos advice unless there’s an unsuccessful appeal, highlighting the limits on Decision Maker’s independence – and that National Insurance numbers are used to merge the three data sets; and page 8 details the activity in the lead up to publishing the data.
  • The uses and users document again highlights the three different data sets used. The section entitled Known Uses of the ESA-WCA Statistical Bulletin includes the sentence ‘Providing the evidence base for assessing the potential effect of policy options’, and the section entitled Valid Uses of the ESA-WCA Statistical Bulletin includes the sentence ‘The data is predominately used for Employment and Support Allowance (ESA) Policy Analysis.’ This reinforces my argument that the key performance indicator for the WCA should be the number of successful appeals – something that is particularly relevant to the debate around the use of audio recordings. Unfortunately the section entitled Areas of Unmet Need doesn’t mention reconsideration statistics or appeals statistics for periodic and IB reassessments – in my view it really should. The section entitled How to comment and get involved makes it appear that they’re very open to user-feedback.
  • The quality statement compares the WCA statistics against nine ‘European standards for quality reporting’. Again it highlights the fact that the the bulletin is compiled from three data sets. There is some discussion around the HMCTS data that suggests the data on appeals against Fit for Work decisions is more reliable than those against Work-related Activity Group decisions. On pages 9 and 14 it highlights the fact that all three datasets are available five months in arrears, despite them not being compiled for another four months – I discuss the delays in publishing here. Page 16 reveals that the bulletin requires 0.5 members of full time staff to produce.
  • The background information is essentially the same document that accompanies the statistics themselves minus the summaries of new data.