https://dataingovernment.blog.gov.uk/claim-personal-independence-payment-service-assessment/
Claim Personal Independence Payment - Service Assessment
Personal Independence Payment (PIP) helps with some of the extra costs caused by long-term ill-health or a disability for those aged 16 to 64. Claim PIP is a digital service that will transform the way users claim PIP, combining telephone application and written application and evidence provision into a single service.
Department / Agency:
DWP
Date of Original Assessment:
19/05/2015
Date of Reassessment:
29/06/2015
Assessment Stage:
alpha
Result of Original Assessment:
Not Pass
Result of Reassessment:
Pass
Lead Assessor:
N. Williams
Service Manager:
A. Holmes
Digital Leader:
K. Cunnington
Reassessment Report
29th June 2015
The Claim PIP service has been reviewed against the 3 points of the Digital by Default Service Standard that were not passed at the original assessment.
After consideration, the assessment panel have concluded that the Claim PIP service is on track to meet the Digital by Default Service Standard at this early stage of development.
Reasons
The service team have addressed the user research issues that were raised at the initial assessment on 8 May in relation to points 1 and 20. The panel were pleased to learn that:
- The team have overcome internal barriers to in-home contextual research, and begun a programme of in-home research with 6 interviews planned per sprint.
- The team have adopted a systematic and practical approach to representative coverage of the complex audience for this product, focusing on 12 varied and challenging conditions which cover a wide range of design challenges that the service must address, combined with more informal pop-up research to include people with other conditions.
- Research has taken place with ‘friends and family’ also present, based on whether the user would normally complete this kind of application on their own or with support.
- The lab research has been observed by a range of DWP staff, including PIP assessors. The panel felt that the feedback loop that this has established was very constructive and should be continued as the team move forwards into beta.
Overall, the team's research plans for the project appear appropriate for the scale and complexity of the audience and the design challenge.
The panel also saw sufficient evidence that users can complete the service unaided and that this is yielding usable outputs for assessors. The 'scenario' based approach to questioning has been improved and 30 more users have completed the service. Users appear to understand this style of question more easily than they did the previous form, leading to better answers. The team now has a clearer understanding of the amount of time it takes users to complete the service (average of approximately 45 mins) and is aware of the current difficulties within the service, which the team are keen to iterate and improve.
Recommendations
The panel recommend that the team continue resolving the pain points within the service. The team are currently working on some of these, including the progressive disclosure patterns, a save and resume pattern and the in-form navigation design, and the ‘review details’ page.
The panel recommend that the team persists with its current research approach - with the representative subset of the audience that has been identified, and the continued involvement of wider DWP colleagues. There will be much more to discover, and more design changes to make, which the panel will look forward to reviewing at the next assessment.
Summary of Original Report
19th May 2015
After consideration the assessment panel has concluded that the Claim PIP service is not yet on track to meet the Digital by Default Service Standard at this early stage of development.
Reasons
The service currently falls short against three points of the standard, as detailed below. There is more work to do in these areas before the panel could confidently say Claim PIP is on track to meet the Digital by Default Service Standard.
Underlying all three points is a need for deeper and broader user research. While the service team have developed a good, convincing and evidenced set of high level user needs (drawn from a large body of prior research) and established that the project is viable and worthwhile, as yet it is not proven that a digital service backed with assisted digital support can address these needs.
Given the nature of this service, and the extensive range of physical, cognitive and psychiatric conditions that could impact the design in many different and unpredictable ways, the bar needs to be set particularly high for user research and assisted digital provision for this service.
Point 1 - Understand user needs. Research to develop a deep knowledge of who the service users are and what that means for digital and assisted digital service design.
The research conducted in the alpha was limited. The single researcher working on the project completed a commendable amount of work during the short time available (and in spite of purdah constraints), but it was nevertheless insufficient to understand this diverse and complex user base. The notable gaps in the research were:
- The end-to-end prototype journey was tested with just 10 users.
- There was no significant or detailed exploration of how the diverse range of conditions in the audience will impact on the design (or designs).
- There was no contextual or in-home research to explore completion of a digital application in the environment in which that application would actually be completed (which is likely to be of some significance given the nature of the users).
- There were no paired research sessions to explore ’friends and family’ involvement.
Point 9 - Create a service that is simple and intuitive enough that users succeed first time, unaided.
With the limited lab testing sample, the overall research conclusion was that none of the respondents would have been able to complete an accurate or sufficient application to allow their case for PIP to be properly assessed (users can complete the service but their answers are deficient). The alpha was therefore unable to prove that the approach in the current prototype meets the user needs in this audience, or provides a viable way forward for the product. We believe a solution can be found; the team will need to spend more time designing and testing options and challenge harder the underlying assumption of a single, long questionnaire equivalent to the existing paper-based form.
Point 20 - Put a plan in place for ongoing user research and usability testing to continuously seek feedback from users.
The research proposals for beta are appropriate, but they do not go far enough. The team need to address the internal DWP barriers to in-home and contextual product research conducted by DWP staff. If they cannot be overcome, there may be a need to consider using external researchers, or locating respondents with relevant conditions who are not customers (i.e. who have not applied and are not proposing to apply). The aim should be to build a significant body of data based on contextual research to sit alongside the data from lab-based usability testing and remote research.
The research plan should look to segment the sample at a more granular level against the various conditions (or their effects, if this is a more appropriate approach) and systematically recruit against this segmentation. Friends and family should be included appropriately. And more generally, greater numbers of respondents are needed, to more thoroughly encompass the diversity of this audience.
There may be a need to increase the number of researchers on the team so that the expanded research activity that this product requires can be delivered.
Recommendations
In addition to the above points, the panel suggest that you develop your plans for coding in the open. Though not necessary for the alpha reassessment, it will be important for the beta assessment that you have released all of your code unless there are robust and convincing arguments for not doing so.
Assumptions on assisted digital funding need to be more fully fleshed out, as according to your research report around 70% of existing Disability Living Allowance (DLA) claimants were helped to complete their DLA form by third-parties, including charities and support organisations. Given the complexity, length and invasive nature of the service, user research for the Claim PIP service might show that support routes such as these are required to meet user needs. If so, the service team must ensure that this support provision will be sustainably funded (including any support not delivered by government).
Summary
The panel were impressed by the team, the passion for the service and its users, the agility in adapting methods, and the amount of work the team has done in the short time the project has been running. The service is a particularly tough research and design challenge given the user base. The panel have every confidence that the team can address these issues and pass a reassessment, given more time.
Digital by Default Service Standard criteria
| Criteria | Passed | Criteria | Passed |
| 1 | Yes | 2 | Yes |
| 3 | Yes | 4 | Yes |
| 5 | Yes | 6 | Yes |
| 7 | Yes | 8 | Yes |
| 9 | Yes | 10 | Yes |
| 11 | Yes | 12 | Yes |
| 13 | Yes | 14 | Yes |
| 15 | Yes | 16 | Yes |
| 17 | Yes | 18 | Yes |
| 19 | Yes | 20 | Yes |
| 21 | Yes | 22 | Yes |
| 23 | Yes | 24 | Yes |
| 25 | Yes | 26 | Yes |
