Audio recordings

Another issue that has been raised is whether facilities for audio recording assessments should be provided. I held a debate on this issue on 12 June 2013 and this page closely follows the structure of my speech.

Assertions are often made about some of the questions asked and the attitude of Atos assessors. For example I recently met a constituent – by no stretch of the imagination is she a disability activist – who told me that the assessor made a comment about her handbag, saying “Well I couldn’t afford that, even on my salary.” Regardless of the fact it was a present, she felt the comment was irrelevant and carried the implication that she did not need the benefit.

Such assertions are regularly denied by Atos and not accepted by DWP Ministers. There are even differences of opinion on the Work and Pensions Select Committee, on which I sit, with some members feeling that campaigning organisations exaggerate such claims.

Audio recording of assessments would allow such disputes to be settled once and for all. For a start they would provide new evidence in the event of appeals. But, more importantly, I would argue they would prompt assessors to ensure that their work was of the highest possible standard – for example they might take more time, ask open as opposed to closed questions, and probe for possible follow-up issues. By improving the quality of assessments in this way, more decisions would be ‘right first time’ and the number of appeals would be reduced. While it might cost money to purchase recording devices and storage facilities, this could well be made up for by a reduction in the number of appeals.

Professor Harrington’s first independent review of the WCA recommended that Atos should undertake a pilot to determine whether audio recordings improve the quality of assessments. The pilot went ahead in Atos’s Newcastle assessment centre between March and May 2011, and an evaluation report was submitted to the DWP on 4 June 2011.

Level of demand
In a Westminster Hall debate on 1 February 2012, the previous Minister, the right hon. Member for Epsom and Ewell (Chris Grayling), set out the Government’s position. He said that due to a lack of demand, audio recording would not be rolled out for all assessments. Specifically he said:

We decided not to implement universal recording because, based on the trial experience, people did not want it.

Unfortunately this assertion is not justified. The Atos pilot concluded that:

68% of customers agreed to the recording when contacted by telephone prior to the appointment.

Due to some claimants not turning up for their assessment, or eventually deciding that they did not want a recording, the figure for those whose assessments were recorded dropped to 46%. But that figure is still substantial, and the demand for audio recordings is reflected in one of Atos’s key conclusions, which stated:

Our recommendation would be that recording should be become routine as it is in a call centre or, for example, NHS Direct.

Parliamentary questions and freedom of information requests have yielded another metric to defend the Government’s position—namely, that only 1% of the claimants in the pilot requested a copy of their recording (this was referred to by current Minister Mark Hoban both at the DWP Select Committee on 21 November 2012 and in a fairly inconsequential follow up letter here). However this cannot be regarded as an accurate reflection of demand for two reasons.

First, assessors in the pilot used hand-held devices and the recordings had to be transferred to computers and burnt to CDs after the assessments. That meant that claimants could not pick up their recording on the day but had to go to the added effort of making a request in writing. In effect, that required claimants to opt into the pilot and then opt in again to get their recording. We also do not know what the claimants thought the pilot was about. Often, when we phone helplines, we are told on a recorded message that the call will be recorded for staff training purposes. It is possible that the claimants in the pilot were not clear about its purpose.

Secondly, claimants were told that recordings would be of use to them only in the event of an appeal. Given that the report was completed just days after the pilot concluded, most of those involved would not yet have received a decision on their claim, let alone come to a view on whether they would appeal. Demand for copies might well have been higher had this metric been measured after a longer period.

Further detail on both of these issues can be found in a very helpful report produced by P V Sutton published in May 2013.

In my debate I therefore asked Mark Hoban to accept that the number of claimants in the pilot who requested a copy of their recording was not an accurate reflection of demand, and that the number of people acquiescing to their assessment being recorded would have been a more appropriate metric to use. Unfortunately, as blogger Mike Sivier notes here, he was particularly obstinate on this point:

Mr Hoban: The hon. Lady has raised concerns about the metrics we use when considering demand for audio recording. We feel that the metrics used are key in showing the exact demand during the pilot.

Sheila Gilmore: Perhaps the Minister was about to come to this point, but I am sorry that he has chosen simply to repeat the 1% figure without addressing the criticisms that have been raised—I have heard them from others, too. The context of the pilot made it difficult for people to get a copy and the pilot was then evaluated very quickly.

Mr Hoban: I do not think that it was that difficult to get hold of a copy. The recording might need to be held on a handheld device before it is transferred to a computer and a transcript is printed, but that does not stop people asking for a copy. I thought that was one point in the hon. Lady’s thoughtful speech that was not well substantiated.

On a related note in my debate Mark Hoban also tried to make a more general point about claimants not finding audio recordings helpful. Again V P Sutton’s report addresses this.

What is happening now
Turning to what has happened in the two years since the pilot, it’s worthwhile referring back to the statement given by the previous Minister Chris Grayling on 1 February 2012. In addition to claiming that there had not been much demand for audio recordings, he said:

We will offer everyone who wants it the opportunity to have their session recorded.

In practice, however, it is very hard for anyone to have an assessment recorded. The current Minister Mark Hoban claims only 9 requests for recordings have even been turned down, as shown in this answer from 6 June 2013, but I believe this grossly underestimates the figures. In the debate Hoban noted that since September 2011 around 4,000 claimants have requested a recorded assessment but Atos has conducted only around 2,000 (sources here and here are also useful in this respect).

For a start recordings must be requested in advance, as emphasised in answers to written parliamentary questions given on 21 May and 5 July 2012. Ironically, as this internal DWP document shows, this doesn’t appear to apply to interviews with regular Jobcentre Plus staff or, as this Daily Mail interview from 12 June 2013 with Consumer Minister Jo Swinson suggests, policy in other areas of Government.

Secondly, as my colleague Stephen Timms said in a debate on 4 September, even when requests are made they are not always met due to a lack of equipment. An FOI response from 22 May 2013 indicated that Atos only has 50 audio recording machines to cover the 11,000 assessments undertaken across the country every week. While officially clients are allowed to use their own equipment, the restrictions on what sort of devices can be used – as this answer from 6 June 2013 shows – render this impractical (something which Mark Hoban actually acknowledged in my debate). Mike Sivier actually suggests that recordings could quite easily be made on Atos computers – I intend to follow this up in the future. Until recently Atos could require a claimant who had requested a recording to undergo an unrecorded assessment if, by not doing so, their assessment would be delayed by more than four weeks. Thankfully this provision has now been suspended,as highlighted in this answer from 6 June 2013.

And thirdly another FOI request from 23 May suggests that the current national roll-out is actually only another pilot that will end later this year (interestingly Ministers have indicated that the results of this will inform decisions about whether Personal Independence Payment assessments should be recorded).

However the biggest barrier to recordings, at least at present, is the lack of information on this option in any official DWP communications to claimants. If people do not know that that service is available, they will hardly ask for it. The DWP website was updated on 6 June and states that ‘We are working to introduce more widespread information for all claimants as soon as possible.’ It is very disappointing that such simple information has not been integrated into communications over the last two years. Unfortunately when I raised this both this issue and the length of the current pilot in my debate, Mark Hoban chose only respond to the latter point and not the former:

Mr Hoban: As has been rightly pointed out, we cannot get a true comparison until we routinely let people know about its availability. I am pleased to say that we are therefore taking steps to boost awareness of audio recording.

The Department and Atos are in the process of amending written communications to claimants by updating the WCA AL1C form. The document is sent to claimants when they need to arrange a face-to-face assessment and will provide more information on how to arrange an audio-recorded assessment. We expect the revised form to be sent out to claimants by the end of next month, once the necessary changes have been made and the form has been cleared for use.

Sheila Gilmore: Perhaps the Minister might be able to explain why it has taken nearly two years to make that amendment. If I understood him correctly, he said that the evaluation of all this process was being extended to the end of the summer, so if the revised letter is not going out until the end of this month or the end of next month, there will be very little time to judge whether that has made any difference.

Mr Hoban: The volume of people going through the WCA on a monthly basis is significant—I believe that 100,000 claims are made for ESA every month—so it will not take long to find out the take-up rate, although we need to make sure that the pilot has the right amount of time to gather sufficient evidence. Earlier the hon. Lady was arguing in favour of a shorter pilot and now she is potentially arguing for a longer pilot in order to get the evaluation right, but she makes an important point.

In addition to the letter I mentioned, the Department has recently provided more information about the audio-recording facility on the “Inside Government” section of the website. By ensuring that more people are aware of the facility we will get a much better picture of how many people are applying for an audio recording and a better assessment of the level of demand.

Level of successful appeals
The report from Professor Harrington in 2010 prompted the Newcastle pilot, and it is worth looking at what he has had to say on this issue since then. In his third report in December 2012 report, which was his third and final one, he said:

The pilot of audio recording of assessments has also been subject to much debate…The Review has seen little evidence from the DWP evaluation of the audio recording pilot of 2011 that the universal audio recording of assessments would improve their quality…further monitoring and evaluation work needs to be completed before a decision can be made.

The Minister might like to interpret Harrington’s reference to “little evidence” as suggesting that audio recordings make no difference, but I would argue that what he was getting at was the inadequacy of the pilot commissioned and accepted by the DWP, which was why he called for more examination of the issue.

The methodology Atos used in their report is highly questionable. What the authors in this pilot did was to take a small number of reports, review them in light of the recordings and conclude that they tallied with each other – that what the written report said and what the recording said were the same. Subsequently, to justify their policy, the main arguments from the Government have both highlighted and ignored the various metrics of demand mentioned in the report. Neither of those approaches answers the key question: do audio recordings improve the quality of assessments?

Instead, I would contend that the key performance indicator for the work capability assessment should be the proportion of decisions that are subsequently overturned on appeal, currently running at 9% (the Government almost acknowledge this in a series of documents published on the production of ESA statistics that I discuss here). A more robust pilot would have involved taking larger samples of both recorded and unrecorded assessments and examining the proportion of successful appeals for both. If they were the same, it would have been fair to conclude the recordings make no difference; but if there were a smaller proportion of successful appeals from those that were recorded, it would be equally fair to conclude that they were worth while.

I believe Ministers should commission their civil servants – not Atos – to rerun the pilot so we can find out once and for all if they reduce the likelihood of fit for work decisions being successfully appealed. If it does then they should be rolled out nationwide.

Letter to Mark Hoban
This is one of a number of points that I made in a letter to Mark Hoban following my debate. I received a response dated 24 July. The Minister enclosed a copy of the updated form informing claimants that they can have their assessment recorded – please see a copy here.

Despite me explicitly explaining that the level of demand was calculated too early as most of those involved would not have received a decision, the Minister glibly responds that only a handful requested a copy.

Interestingly Mark Hoban does commit to looking further at examining the levels of successful appeals in determining whether recordings improve the quality of assessments. Although he notes that tribunals overturn decisions for a variety of reasons, emphasising that often new evidence is presented that wasn’t available initially, I would argue that this could in part be down to Atos assessors not asking sufficiently probing questions or engaging sufficiently with claimants.

I remain sceptical about the proposed timescales for the ‘evaluation’ and find the comment that they cannot yet say what they would be evaluating rather strange. Normally one would expect thereto be an evaluation framework at the outset! Once the results of the national pilot are published I will seek to raise this issue in the chamber once more.

Finally I’m concerned that the wording in the information leaflet – with its emphasis on claimants having ‘no legal right’ to a recording – is quite off-putting. This might suppress take-up and allow Ministers to claim that there is no demand for their roll-out.

Update: 18 September 2013
Since I received a reply from Mark Hoban I’ve come across this document setting out the Government’s official policy. I’ve also been made aware of a number of further points.

Claimants’ rights
This internal DWP-Atos document from late 2010, obtained under the Freedom of Information Act, highlights legal advice to DWP that recording of assessments for the claimant must be offerred without unreasonable obstructions, and that the majority of claimants do not have the financial means to provide specialised recording equipment themselves.

According to this Freedom of Information request, Atos assessors should no longer be refusing to record assessments as it has been part of their role description since 1 December 2012. Similarly this internal Atos document suggests that, due to the Data Protecion Act, assessors should not be able to stop claimants recording their assessments.

Promoting recordings
It has been pointed out to me that the AL1C form is not the main point at which claimants will learn about their assessment. This will come when Atos operatives arrange them over the phone. I would therefore suggest that the opportunity to have assessments recorded should be flagged up at this point.

This same individual pointed out that claimants can only request one change of appointment following this initial phone call, and that requesting a recording would count in this regard – this is far from satisfactory. See this email which confirms this is the case.

I acknowledge that recordings could be promoted on ESA50 forms, but the DWP’s response would likely be that not all claimants are called for assessment.

Cost of recordings
The answer to Question 10 in this set of answers – again a Freedom of Information request – suggests that any costs of additional equipment so as to undertake audio recordings would be borne by Atos and not the taxpayer.

Recording Equipment
Current guidance suggests that claimants must use dual tape or dual CD recording machines to record their assessments. However:

  • The initial Newcastle pilot used a single hand-held digital recorder, with a copy being made on a laptop afterwards.
  • Dual recording machines cost a minimum of £500 and the leading manufacturer – Neal – doesn’t generally sell to individual consumers.
  • This Freedom of Information request suggests that claimants can use two dictaphones simultaneously.

Pilot length
I have received an answer to a written parliamentary question that indicates that the current pilot period will last until the end of 2013, but that no changes will be made until an evaluation report is published.

Update: 15 January 2014
The fourth independent review of the WCA by Dr Paul Litchfield contains the following extract on audio recordings:

35. Of all of the recommendations made by Professor Harrington in his three reviews, the one that attracted the most comment in this Call for Evidence was Recommendation 8 from the first review, concerning audio recording of assessments: ‘The review recommends that Atos pilot the audio recording of assessments to determine whether such an approach is helpful for claimants and improves the quality of assessments.’

36. Call for Evidence responses included: “Record as standard every face to face. Always give a copy of recording to the client. This way it is very easy – if a reconsideration is required the DM can just pull up the audio recording to see what actually was said during the assessment.” Individual respondent, Ms C “The recordings might in fact assist both DWP and ATOS to rebut challenges and complaints regarding the contents of the ESA85 reports.” Disability Solutions West Midlands

37. After the results of an initial inconclusive pilot conducted by Atos Healthcare in June 2011, the Department continued to offer audio recording although this was not widely publicised. The Department asked Atos Healthcare to try to accommodate requests for audio recordings within four weeks. Where this was not possible the assessment would go ahead without a recording. The Department has now removed the four week deadline.

38. The guidance leaflet (WCA AL1C) sent to people making a claim was amended in August 2013 to publicise the option to request an audio recording. This should mean that people attending a face to face assessment are now aware of this option and that everybody who requests a recording should be provided with one. The recommendation is therefore fully implemented.

I find this slightly disappointing. As I said earlier, the methodology of the Government’s research thus far has meant it has been impossible to tell whether audio recordings improve the process. In addition, given that the current roll-out of audio recordings is also described as a pilot, the option to have a recording isn’t yet mentioned in the assessment arrangement phone call, and the issues people are still experiencing with securing a recording, I would have thought it would have made more sense to wait until the outcome of the current pilot before declaring Professor Harrington’s previous recommendation as ‘fully implemented’.