Back in March we launched a beta version of the Government Service Design Manual. As the product manager at the time, I remember one of the things the team agonised about was whether we should launch any public version of the manual without search functionality. The manual already contained a lot of content then - over 150 guides - and users repeatedly requested tools to help them search the material for what they wanted.
In the end, search was added for the live version we released in April, rather than in the beta. But six months on and with the benefit of hindsight - and data - it’s interesting to ask whether we were right to worry about it so much. It’s also a good check of the reliability of user needs as a guide for prioritising a product build. Did people get as much value out of search as they assured us they would?
Does search get used?
A cursory glance at the analytics suggests that maybe all that effort wasn’t worth it after all. Since April 16 (when the beta sticker was taken off the manual) 5,881 visits to the manual used the site search facility. That’s a little under 4% of the 150,000 or so visits the site has received from then - 3.92% to be more precise. That doesn’t seem like a very big number; less than one in twenty visits used the search tool as a navigational aid.
However, when compared to the wider GOV.UK site, that figure doesn’t appear quite so bad. 4% of all GOV.UK visits use site search (with a total of 1.8 million visits using search in October) so the use of search on the manual is typical for the domain as a whole.
Who uses search?
This picture gets more interesting when we examine the behaviour of site search users in more detail. One way of determining whether adding a feature to a product has been successful is by demonstrating that it has helped that product’s intended users to achieve their needs. In the case of the manual, we wanted our users to be able to find what they wanted and to engage with the material. There are two proxy measures we can use to investigate this; the number of pages per visit, or the length of time per visit.
For both of these metrics, site search has a dramatic effect. On average, visitors who use site search spent a full 7 minutes longer on the site (10:21 versus 3:21) and look at 6 more pages (10.11 versus 4.07) compared with users who don’t use the search box.
Correlation doesn’t imply causation - we don’t know for sure that site search made users more engaged, or whether search was a feature used by people who were more interested in the first place. Either way, it’s good news for us. Search has either made the experience of using the manual better, or been a useful feature for those who use it most.
What's happening now?
Now the manual has been in use for a little over six months, we’ve taken the opportunity to undertake some more user research and take a closer look at the analytics. Working closely with service managers across government, we have gathered a range of feedback. Users want more reusable resources they can access, case studies about how other services are being built and clearer guidance which relates to the service standard points. We’re now developing a plan to take those points forward, so keep watching the GDS blogs for updates.
