https://dataingovernment.blog.gov.uk/emergency-travel-documents-service-service-assessment/
Emergency Travel Documents Service - Service Assessment
An emergency travel document (ETD) enables you to get back to the UK or your country of residence from wherever you are if your passport is not available for one reason or another. It is only usable for a defined journey which is written into the document: when you get back, the document is withheld by Border Force and later destroyed by the Passport Office (HMPO). It is more expensive than a passport and so is usually for emergencies. The service has a very wide user base, including:
- people who have lost their passport when travelling
- expatriates who have to return home urgently and find their passport has expired
- children of expatriates who have never had a British passport but need to travel urgently to the UK
- prisoners about to be expelled from a country
- people involved in a crisis situation abroad
Department / Agency:
FCO
Date of Original Assessment:
21/07/2015
Date of Reassessment:
21/09/2015
Assessment Stage:
alpha
Result of Original Assessment:
Not Pass
Result of Reassessment:
Pass
Lead Assessor:
L.Scott (Original) / M. Knight (Reassessment)
Service Manager:
R. Sayce
Digital Leader:
A. Daniels
Reassessment Report
21st September 2015
The Emergency Travel Documents Service has been reviewed against points 1, 2, 3, 6, 12 and 14 of the Service Standard which were not passed at the original assessment.
After consideration the assessment panel has concluded that the Emergency Travel Documents should proceed to private beta.
Reasons
The service does not yet meet point 3 of the standard because some key roles were missing from the team currently delivering the service. The panel believes that the service is now ready to proceed to gather more feedback from a limited private beta, in line with the conditions set out below.
The service team presented the details of the recent assisted digital user research done with the Age Concern centre close to Alicante in Spain. The panel were impressed with the efforts that the team has made to identify user needs for support, and to test the service with users who are, by the nature of the service, not UK based at the time of use. The team are participating in a cross-government group to share findings and identify best practice for researching assisted digital users overseas, which will contribute towards greater consistency for British users needing government support from abroad. It was noticeable that the assisted digital work done had delivered a better service for all users, not just assisted digital users, and the panel would like to encourage the team to continue this good work in the next phase.
Some concerns remain over the way the team had approached the alpha. The aim in alpha stage should be to prototype and explore approaches to meeting user needs, rather than to produce production ready code and functionality. The panel were also concerned at the balance of the team. In particular, the panel was concerned about the way that development resource was prioritised, and other important skill sets were not present throughout the alpha (for example, content design, design and user research).
The panel looks forward to seeing some of the improvements mentioned in the assessment (for example, the removal of the need to enter duplicate information to book an appointment at the end of the flow). As the team move forward into beta, the panel would encourage the team to continue to test and iterate the current journey in addition to adding new features, as well as balancing the skills available to the team.
Recommendations
The service should address the following recommendations ahead of the beta assessment.
Private beta
- The private beta must be limited in scope by users and time, with an overall plan to be agreed with GDS before entering the private beta.
User research
- The team should create a plan for future user research, including assisted digital research. This should include face to face research in the UK with potential users to compliment the existing WhatUsersDo work. Participants must include less experienced travellers, people with low and no digital skills who are likely to need assistance, and people with a range of disabilities and access needs. The research must cover finding the new service, and test the service on the device the potential user is likely to have access to while travelling.
- The team must carefully test any support for ‘proxy’ applications before including it in the private or public beta. This could introduce significant confusion into an otherwise simple service.
- In addition to completing and acting on the planned research with Age Concern in Alicante, we recommend the team does research in countries where access to digital services can be more problematic, for example by continuing with the plans for sessions in Addis Ababa and Islamabad.
- At this early stage, the team is considering different design options for assisted digital support, largely based on existing support through consulates and contact centres. As the service develops, the team must demonstrate how support is being designed and iterated to meet user needs.
The team
- The Service Manager should prioritise the recruitment of design and content design resource for the team for private beta and future phases. Borrowing patterns from other services or having a designer ‘look in’ on the service are not substitutes for these necessary skill sets, and this will become more important in future phases. These designers should actively participate in user research.
Tools and systems
- The team should reconsider their decision to send personal data by email in the clear; sending this by email poses a risk to the security of the data. The team should instead consider sending a receipt only.
- The team should note that the GOV.UK APIs that they rely on are unsupported, and as a result may break or change without notice. The service should have plans in place to identify if this happens and also consider what the impact on users might be if the APIs change significantly.
Simple and intuitive service
- The team work on the content of the service with a content designer. Particular issues include poor validation messages, content not to GOV.UK style, and long headings.
- The team review the design feedback document that will be sent separately. Particular issues include frustrating summary screen, validation that can be distracting, and handling of approximate information.
Summary
This is a complex service and it was great to see the work that has gone into developing it. The panel hope that the pass at alpha and the recommendations above encourage the team on their journey towards a beta assessment. The panel look forward to hearing about the private beta when the team return for the beta assessment.
Summary of Original Report
21st July 2015
After consideration the assessment panel has concluded that the Emergency Travel Documents Service is not yet on track to meet the Digital Service Standard at this early stage of development.
Reasons
User needs and user research
Point 1 - Understand user needs. Research to develop a deep knowledge of who the service users are and what that means for the design of the service.
Point 2 - Put a plan in place for ongoing user research and usability testing to continuously seek feedback from users to improve the service.
Point 12 - Create a service that is simple and intuitive enough that users succeed first time.
The service team has identified the top user need for the service (I need to travel on a booked journey without a passport) and have identified improvements to be achieved and current pain points e.g. reducing waiting time in the consulate. The vision for the future service (to apply online, be verified remotely, digital photos, pay online, receive an emergency travel document (ETD) at departure destination) is compelling.
However, the panel could not see how the team had used research and discovery to evidence and validate these needs and pain points. We reviewed the report from IFF, which suggested that users feel reassured attending the consulate, and showed little appetite for a digital service. More research is needed to understand the needs of users, and ensure that the service design meets these.
Relying on remote, scenario-based user research means the team aren’t exposed to the needs of their genuine users, and that the users doing the testing are not fully engaged with the service (e.g. where they select a country at random).
The team have also not researched specifically with lower-skilled or lower-confidence users, or those with assisted digital needs. As such the prototype service lacked informed assisted digital support routes to test and iterate, instead relying on users requiring assisted digital support to use the inferior paper service.
The alpha is the time to get a deep understanding of users and their needs, and the landscape for transforming the digital service. The service team has spent much of the alpha building the real service, missing the objective of an alpha. The service team hasn’t used the alpha to explore many of the identified user needs.
The prototype demonstrated seemed to focus on feature completeness rather than building something that would help the team learn about their users. The panel were unclear why many paths of the journey were built if they weren’t being tested at alpha. The panel would recommend the service team investigate using the GOV.UK prototyping kit. This would deliver a more functional prototype that is more realistic than the client-side javascript solution demoed.
The prototype does not yet include the most complex elements, such as payments and photo upload. The team had surveyed previous users and had an understanding that digital confidence decreased when abroad, with particular concerns around the potential data costs of completing a form online.
The team has iterated the prototype frequently, although much of this was addressing smaller content changes. Many identified needs have been left for beta development. The team has identified some user groups to engage with in beta, e.g. farmers in Africa and expats in Spain. We’d encourage far more of a focus on non-scenario based research. A user researcher joining the team is essential.
The team were concerned that the service planned to stop using the prototype and only use production code going forward. Prototyping and testing regularly with users is an important part of the process for the entire development of a service. Whilst testing with production code may appear to save time, it increases the risk of building the wrong thing, and it slows down the time taken to iterate changes for user research.
Significant portions of the service overlap with two existing services - passport renewals and lost and stolen passports. The panel would have liked to have seen more evidence of the team having engaged with these existing services and incorporating their findings from user research. The service team mentioned they had engaged in trying to share code, but at alpha stage learning about existing research and design patterns would be more valuable.
The team
Point 3 - Put in place a sustainable multidisciplinary team that can design, build and operate the service, led by a suitably skilled and senior service manager with decision-making responsibility.
Understandably for a small team, there are many overlapping roles. There are however key roles (including design, content design, user research and data analysis) that are not represented on the team, with responsibility being shared for theses between the product manager (FCO) and the business analyst (supplier side).
A user researcher, working at least 3 days a week, is currently missing, and is a vital role on a service team. This would reduce the reliance on an outsourced user research company and help address some of the concerns the panel had around the research methodologies used. Having a user researcher on the team would have helped the service team better target their research in alpha.
Currently a content editor at FCO reviews the content. A content designer should be working more closely with the service team to design content to ensure the service meets user needs, rather than providing a proof read at the end of the process. The service has particular challenges around supporting applications from people applying on another person’s behalf - we recommend further research in this area.
The service uses the GDS design patterns and toolkit, however there are small inconsistencies that will need to be addressed. The panel will send through design recommendations separately, as well as a review of the service’s content.
Technology
Point 6 - Evaluate what tools and systems will be used to build, host, operate and measure the service, and how to procure them.
The panel believes the front-end of this application is over-engineered. The team should reconsider the technology choices used and build for progressive enhancement. For example, the use of an isomorphic front-end complicates the build and will make it more difficult to iterate.
The majority of the journey could be delivered as HTML, with JavaScript used to enhance aspects (e.g. validation). There is no need to deliver the entire journey using JavaScript. No allowance was made for users who have JavaScript enabled but don’t receive it.
The architecture includes a Scala backend. The choice of language itself is not unreasonable in this case, but Scala is a very difficult skill to recruit for and makes it more difficult to move from an incumbent supplier, so the team should weigh this against potential recruitment problems.
There are a number of services that need to be called in order to complete a transaction or submit an application, e.g. create a PDF, send an email, insert into the case management system. There is no plan to keep data consistent between these services or deal with a partial failure. If one of the services fails this has an affect on the whole transaction, e.g. the case management system fails but the email confirmation succeeds. This needs to be addressed.
At the moment there is no data store on the server and it is important to address audit of applications. Mismatches between the case management system and the web front-end will otherwise be very difficult to identify.
Digital take-up
Point 14 - Encourage all users to use the digital service (with assisted digital support if required), alongside an appropriate plan to phase out non-digital channels/services.
The panel did not hear a compelling reason for the lack of a plan to increase digital take-up to 100%. In particular, it was not clear why there shouldn’t be an ambition to remove the paper channel (where anyone needing help accessing the digital service would be supported via the assisted digital channel, and understanding that paper is not an appropriate assisted digital support route).
Recommendations
The panel recommends that the service address the following:
- Hire a user researcher to work alongside the service team.
- Carry out research with actual end users of this specific service. We recommend asking users who are already in the consulate waiting for their documents to be processed.
- Research all user journeys, including the least happy path.
- Using appropriate recruitment methods, carry out research with users with all levels of digital skills and confidence (including those who would seek support from third parties or friends and family) to inform the design of both the on-screen service and any assisted digital support.
- Ensure ongoing research to account for extra service complexity as new features are added.
- Collaborate with the Home Office to learn from the user research carried out for the ‘lost and stolen passport’ service.
- Hire a content designer to work alongside the service team.
- The panel were concerned about mailing large amounts of personal data being sent between the embassy and to the recipient in the clear. The panel recommend sending notifications only and that users log in to get the data. The panel noted that this step is short term until the case management system is in place. It might be better to mock the interface to the case management system for testing and omit the email stage.
- The session caching server will hold very sensitive data, by default the server is designed to exist within a trusted environment. Security around this store needs to be defined.
- Reconsider the technology choices for the front end. Build for progressive enhancement. The team should discuss this further with GDS.
- Consider using the GOV.UK prototyping kit for future prototypes.
- Consider the licence you need to open your source code.
- Establish a plan to achieve 100% digital take-up.
Summary
There are positives to the work the team has done so far, for example, the team showed empathy with the distress many users would be experiencing, especially if they were in need of an emergency travel document as a victim of crime, and as mentioned earlier the vision for the future service is compelling.
It was also positive to see the collaboration in the team and to hear how all team members understood the value of their work and how it relates to the overall vision, and were able to contribute ideas and suggest change.
The panel were pleased to hear that the team have already put a lot of thought into how they will measure success, and are speaking to the Performance Platform to share data in the open.
However, as detailed above there are a number of areas where the team should carry out further work, in the alpha stage, to ensure that the service is well positioned for beta development, and delivers a high quality service which will meet user needs.
Digital Service Standard criteria
| Criteria | Passed | Criteria | Passed |
| 1 | Yes | 2 | Yes |
| 3 | No | 4 | Yes |
| 5 | Yes | 6 | Yes |
| 7 | Yes | 8 | Yes |
| 9 | Yes | 10 | Yes |
| 11 | Yes | 12 | Yes |
| 13 | Yes | 14 | Yes |
| 15 | Yes | 16 | Yes |
| 17 | Yes | 18 | Yes |
