https://dataingovernment.blog.gov.uk/surveillance-2014-service-assessment/
Surveillance 2014 (Animal and Plant Health Authority) - Service Assessment
The service provides a digital channel for Private Veterinary Services (PVS) to submit sample information to APHA and to track reports generated from their submissions and testing requirements. Digital submissions are more readily processed than paper forms, and offer the opportunity to capture more accurate data for disease surveillance.
Department / Agency:
DEFRA / APHA
Date of Original Assessment:
25/11/2014
Date of Reassessment:
19/1/2015
Assessment Stage:
Alpha
Result of Original Assessment:
Not passed
Result of Reassessment:
Pass
Lead Assessor:
L. Scott (Original) / S. Wood (Reassessment)
Service Manager:
K. Line
Digital Leader:
J. Pierce
Reassessment Report
19th January 2015
The Surveillance 2014 service has been reviewed against four points of the Service Standard (10, 13, 17, 18) at the end of alpha development.
Outcome of service reassessment
After consideration the assessment panel has concluded that the Surveillance 2014 service is on track to meet the Digital by Default Service Standard at this early stage of development.
Reasons
Point 10 - Put appropriate assisted digital support in place that’s aimed towards those who genuinely need it.
The team had identified potential users of assisted digital (AD) support and had developed a good understanding of needs, using research findings to develop an AD persona. The team has a good plan to expand user research in beta and to test assumptions around numbers and likely support required. Initial proposals for support seem appropriate based on findings so far, and the team confirmed that the support would be funded.
The service should continue with their plans for beta; to continue with AD user research, to test their hypothesis, and to start to develop support. The team should also test the digital service with people who have lower confidence and digital skills.
The assessment panel was impressed with the great strides the service team has made in this area. The panel noted with approval the plans to test in trade associations and in smaller practices. It was also understood by the panel that AD will be brought into mainstream user research sessions.
Point 13 - Build a service consistent with the user experience of the rest of GOV.UK by using the design patterns and the style guide.
The assessment panel appreciated that the service team:
- Was hiring an interaction designer to build the front-end using the GOV.UK toolkit.
- Was keeping the content designer on during the beta.
- Recognised that the language used in the alpha (which was consistent with the current paper form) was not necessarily the language of the users, and planned to address this.
- Planned to reorder the information within the service to better match users’ own experience of interacting with a sick animal.
That said, the service has more of the look of a digitised paper form as opposed to GOV.UK service. Indeed, the assessment panel noted that the title itself - Surveillance 2014 - needs to be looked at, if only for the fact that it sounds a year out of date and is not self-explanatory.
Point 17 - Be able to test the end-to-end service in an environment identical to that of the live version on all common browsers and devices. Use dummy accounts and a representative sample of users.
The service team was able to confirm that the service had been tested in GDS recommended browsers. However, this only involved five users using desktop computers. This approach appears to be based on an assumption that users would not want to submit sample information in the field, even if a mobile or tablet alternative was available.
The service demo started after the login process. The login component of the service needs to be demonstrated at the beta assessment, along with the payment process using dummy logins if necessary.
Point 18 - Use analytics tools that collect performance data.
It was noted that the service team currently uses an analyst who sits outside of the core team. The risk is that collecting performance data results in Management Information (MI) reports, at the expense of analysis. The service team is working with the GDS Performance Platform team and that it plans to look beyond the four Key Performance Indicators (KPIs) that GDS recommends.
The panel was especially impressed to hear that one of the KPIs included tracking where users made mistakes when filling in information, with the aim of making the service more usable in future.
Recommendations
The assessment panel recommends that the service:
- Works closely with the GOV.UK design team and takes part in the GDS design community.
- Continues user research into AD to realise the extent to which this is required.
- Expands user research to the veterinary practice staff as often these are the people who enter the data.
- Continues its AD research and, as above, extends this to small veterinary practices and their staff (who might be more likely to be using the new digital service).
- Gives serious consideration to building the beta along responsive design principles (build for mobile and tablet first), i.e. to build for the future. Do not assume that users would always prefer to submit information about samples via a desktop computer.
- Conducts user research with younger users to explore the preferred mobile working practices of frequent smartphone and tablet users.
- Conducts further research into what fields are mandatory and why optional fields are required (if fields are optional, what is the cost to the user if these fields are left blank?).
- Explores how many questions can be removed, or the answers inferred from previous replies ("do the hard work to make it simple").
- Brings an analyst into the core team so that they can take part in sprint planning and research sessions. This will help avoid data being used solely for MI reports, which can be the case if the analyst is separate from the team.
- Improves the scannability of the client address by changing its layout; tests whether users prefer it to appear on every single page.
- Provides species-specific questions, and hides everything not directly relevant to the options already selected (this might help users prioritise the top symptoms).
- Challenges its assumption that existing users will find it reassuring that the service mimics the paper form, as the benefits of this will only exist during the transition period. The longer the time period after the transition, the more new users you'll have who will only be familiar with your online service, and the more the online service will be judged on its own usability.
- Substitutes plain English for technical jargon wherever possible to help users quickly log the information necessary to send their sample (e.g. ‘farm animal health tracking’ for ‘surveillance’, ‘sample information’ for ‘submission’, ‘to help us improve future testing’ for ‘test validation’).
Finally, when the service team returns for the beta assessment it must also demonstrate the:
- Login page
- Payment element
Summary
The service team showed a welcome level of passion and commitment. The team clearly believe in the work they are doing and evidently want to do their best for the users of the service. As the team works in a more commercially competitive environment than most other government services, they will clearly be aware that if new service does not meet with customers’ approval, then they have alternative service providers available to them. The service therefore has a clear requirement to provide users with an excellent service.
A successful alpha review is not a guarantee of success, but it is a clear indicator that a service is on the right track. The panel looks forward to seeing the team again for the service's beta assessment.
Summary of Original Report
25th November 2014
The Surveillance 2014 service has been reviewed against the 26 points of the Service Standard at the end of alpha development.
Outcome of service assessment
After consideration the assessment panel concluded that the Surveillance 2014 service is not yet on track to meet the Digital by Default Service Standard at this early stage of development.
Reasons
The panel decided that the service was not yet at the stage expected to proceed to Beta development. While many things are on track, and the direction seems good, there are a number of criteria on which the service did not pass.
10. Put appropriate assisted digital support in place that’s aimed towards those who genuinely need it.
To move into Beta, the service team needs to have carried out research with assisted digital users, to have a plan for assisted digital in Beta, and to have considered how to provide a sustainable service that’s free for assisted digital users. This requirement is only unnecessary if there is demonstrable proof from user research that there are no users with assisted digital needs.
13. Build a service consistent with the user experience of the rest of GOV.UK by using the design patterns and style guide.
The service needs to look and behave consistently with GOV.UK. It should follow the GOV.UK design patterns and style guide. If the service team feels it has a better approach, or the provided design patterns don’t meet user needs, the service team should contribute to the service design community and suggest a new pattern. The panel would expect an Alpha to more closely resemble a GOV.UK service visually, and much of the microcopy within the Alpha service was not to GOV.UK style.
17. Be able to test the end-to-end service in an environment identical to that of the live version on all common browsers and devices. Use dummy accounts and a representative sample of users.
To move into beta, the core service needs to be working end-to-end. The service team must demonstrate a working prototype that meets user needs and has tested well in user research.
18. Use analytics tools that collect performance data.
To move into beta development, a firm plan for performance data capture must be in place and an analytics package installed. The service team must demonstrate they have the ability to interpret data analytics to identify needs, solve problems and improve the service.
Recommendations
We recommend that the service team focus on the following ahead of alpha reassessment.
- Ensure that a designer, content designer and analyst are available to work with the service team during this early stage of development.
- Complete a fully working prototype to test the core service with users (this could be restricted to the most popular journey).
- Test the service end-to-end, from submission by veterinarians to uploading of results by laboratory staff.
- Make a plan for the service being taken offline and show how this will be tested. Demonstrate how the effect on users will be managed (see Uptime and availability).
- Carry out a full design review using the GDS resources. As discussed earlier, the service doesn’t follow the GOV.UK front-end toolkit (see Design patterns, GOV.UK elements, Design patterns wiki, What your service should look like, Resources for designers).
- Work with the GOV.UK content team to ensure that content and microcopy help the service meet user needs and stick to GDS style (see Resources for content designers).
- Carry out research to identify any users with assisted digital needs (see: Researching assisted digital users).
- If assisted digital users are found, carry out research with those users and plan to carry out regular research during Beta (see: Researching assisted digital users).
- If assisted digital needs are identified, make a plan for how the service will address those needs, to provide a free, sustainable service for assisted digital users (see Assisted digital).
- Choose an analytics package and install it in the service (see Analytics tools).
- Start working straight away with the GDS Performance Platform team to agree extra KPIs to measure success. Start work on a public dashboard to display service performance (see Measurement).
- Open up code as a default.
- Look again at GOV.UK Verify. The service team must show concrete evidence to support not going with GOV.UK Verify. Other services in similar situations, such as the Rural Payments, are using Verify. The fact that many of the service users have a Government Gateway account isn’t a sufficient reason not to migrate (see GOV.UK Verify guidance).
- Investigate ways of avoiding surfacing personal data where possible. For example, when showing the list of clients to a veterinarian consider showing less information. This is not a blocker, rather a suggestion to investigate.
- The notion of data aggregating to IL3 is no longer valid. It would be good to revisit this. The new security classifications should be used (see Security as enabler).
Summary
The assessment panel was really impressed by the agile way the service team is working. This is a great example of a multi-disciplinary, co-located team working together to build a service focused on user needs.
The panel notes the team's commitment to user research, including field research. The team clearly articulated user needs and the whole team were clear on the most important ones to solve first.
The team demonstrated a great understanding of the context in which users work. The panel loved the dedication the team demonstrated to embedding research into sprints and making regular, often radical design changes based on findings. The panel looks forward to seeing how the team applies this same dedication and pace during research with potential assisted digital users, and the internal users in laboratories.
The team had lots of interesting early thoughts about a number of the points raised by the Service Standard. The team's commitment to building a digital service that people prefer to use was clear.
The team's presentation, knowledge and answers to the panels questions were clear and well-prepared.
Digital by Default Service Standard criteria
| Criteria | Passed | Criteria | Passed |
| 1 | Yes | 2 | Yes |
| 3 | Yes | 4 | Yes |
| 5 | Yes | 6 | Yes |
| 7 | Yes | 8 | Yes |
| 9 | Yes | 10 | Yes |
| 11 | Yes | 12 | Yes |
| 13 | Yes | 14 | Yes |
| 15 | Yes | 16 | Yes |
| 17 | Yes | 18 | Yes |
| 19 | Yes | 20 | Yes |
| 21 | Yes | 22 | Yes |
| 23 | Yes | 24 | Yes |
| 25 | Yes | 26 | Yes |
