We’re both passionate about how data can improve how government works, and we want to start a data revolution focusing on service delivery and improving human experience.
We believe government is hugely underestimating the need to shift how we create, use and share data. We want to start a wider conversation about data and build a cross-sector, diverse and inclusive community of interest and practice about data in the public sector.
The problem to be solved
Good quality, user-focused data is fundamental to good decision making, and data can be used to transform the experience of everyone who interacts with government. It forms a part of the government transformation strategy and the centralisation of data policy at DCMS is a strong sign that data is finally being taken seriously.
Although initiatives such as the Data Science Accelerator are welcome and there are some cross-government data and analysis communities forming, there’s a real opportunity to do more. There is not yet a strong practical movement in central government focusing on the applied experience of data for better services - the intersection between digital, data and technology work.
To solve this we want to share experience and move strategy and intent into practical delivery. This will help us make government more efficient and effective, and offer better services to the millions we serve every day.
What is the data revolution?
We think a data revolution across government should:
- focus on user needs and iterative development of data products – this will make sure data is relevant and easy to understand, promoting virtuous cycles in its collection, quality and usefulness
- use advances in technology and ways of working to radically increase the range of government decisions on which data and evidence is brought to bear – empowering frontline colleagues to make data integral to their everyday decisions
Before we get to that point there are 5 things we need to do:
1. Fix the plumbing
We need to treat data like a precious commodity for our sector, as necessary to government’s success as water is to life. And as with water, we should:
- gather it avidly but safely
- store it efficiently
- clean it properly to make it safe to consume
- build the infrastructure to distribute it fairly
- make its consumption part of our daily routine
We should rebalance our efforts to prioritise access to source systems data, to untapped sources like PDFs, free text, voice and others. We should invest in APIs over large systems, in better tagging and metadata, in the basics of interoperability and in making data accessible to the huge range of users who would potentially benefit.
2. Get the right skills
Recruitment efforts are already being made to address the data skills gap in government, but we believe we should take the opportunity to invest in our own people and offer them opportunities to build modern data skills, as well as bringing new expertise in from outside.
We should offer transitions which allow public servants such as analysts to move between professions and use their skill sets to become data specialists. We’ve got thousands of budding data scientists - we just need to invest in them.
Equally important are data leadership skills. We need to train our leaders in open-mindedness and data literacy in the technology age. We need leaders who are able to ask incisive and challenging questions about data projects and teams who are empowered by a clear vision and the right skills and ways of working to make it happen.
3. Fund teams to work across boundaries
This can yield incredible results - the incredible work of the race disparity audit is one example of this.
While perhaps not the most glamorous, it’s important we continue to focus funding on things like getting away from Excel as the default front end (and often backend!) for our data.
We need to escape binding contractual arrangements which mean we cannot get access to our own data and building APIs off legacy systems which will allow an explosion in the potential for better decisions.
Delivery of these unglamorous data fundamentals is time consuming and difficult, and we do not yet have all the solutions. A strong community, funded correctly to focus on sharing and enriching data, can help build understanding of where the real value is, iterate our delivery approach and ultimately direct investment more effectively.
We need to fund more data projects that cut across and ignore organisational boundaries that are outcome-driven, not driven by the needs of a single department or system.
4. Build guide rails
We need to find a way for the distributed network of data custodians, publishers and users to work together in a loosely coupled but co-ordinated way both within and across government departments. One way to work towards this is through shared standards, guidance and patterns.
However, there's currently no equivalent in the world of data for the brilliant government Service Standard (though many of its principles equally apply to data) - meaning there’s no guidance available across our sector for how to do data well.
The Government Digital Service has done by far the best work in getting us as far as we’ve come, via open standards, guidance on APIs and more. But in our experience these are at best applied patchily and at worst ignored. We need to work harder to create and join up great standards with people able to implement them.
5. Collaborate and share our data
A single organisation could never hope to have all the data we need to make the best decisions about the people we serve, so we need to share with and learn from others.
Opening up government data can be scary. We correctly prioritise protecting people’s personal information and after all, it’s easier not to share than to share. But this is not good enough anymore, when we’ve collectively got data which could hugely improve service delivery and efficiency.
We have a responsibility to share more and create common goals and funding models which allow that to happen.
And ‘we’ is not just central government. The true potential for more efficient and effective public services will only be realised when we work across sector boundaries too - with local authorities, charities, think tank and academia. We need to build marketplaces for our data, where data sets and insights can be combined, shared and enriched.
There's already excellent work in this field, including the Open Data Institute, the Office for National Statistics and many others. We want to amplify that and to persuade government to invest far more heavily in opening up our data.
What’s next?
We’d love this post to kick off the beginning of an open, diverse and inclusive, human-centred community of practice around data in the public sector. We’d love your thoughts on:
- this post
- what you think a data revolution might consist of
- the big pain points of working with data
- how a data community might work (meet ups? Events?)
- anyone you think we should talk to
- how to bring together the existing amazing work on data and amplify it
We’re really open to your ideas and feedback. You can contact us in the comments below or on Twitter @kitterati @robinlinacre.
We’d also like to encourage a series of blog posts from different perspectives and different organisations. Over to you, we’d love to hear from you.
You can find out more about the Department for Work and Pensions on the DWP Digital blog.
18 comments
Comment by Tomas Sanchez posted on
Great blog with a powerful message. I really enjoyed the plumbing analogy! One thing to note is that plumbing can only work efficiently if our own plumbing bears some resemblance to our neighbours' plumbing. If I can drop the analogy, data only flows smoothly if the data architecture is planned (to some extent) centrally (https://dataarchitecture.blog.gov.uk/2019/02/26/data-is-not-technology/) . We don't need to centrally understand every detail of every government department, but we can hope to have a general idea of how information is acquired, organised, and archived.
At the ONS we organise a Government Data Architecture Community (GDAC). Our mission is to share ideas, experiences and methods in an effort to standardise the way we work and facilitate collaboration between government departments. We want this to be as representative of as many parts of govt as possible so, if you are not already a part of the community, please get in touch at data.architecture@ons.gov.uk
Comment by Leanne Smith posted on
> Help us start a data revolution for government
How long have we got before you leave government?
Comment by Emma Gordon posted on
Many thanks for this insightful blog. Within the new ESRC-funded Admin. Data Research Partnership (ADRP), we are also passionate about using data to improve how government works, collaborating with departments and devolved administrations to create new linked datasets to inform evidence-based policy decisions. We welcome all opportunities to work with DWP, MoJ and others to achieve this. Find out more about ADRP here:
https://esrc.ukri.org/research/our-research/administrative-data-research-partnership/
Or contact the team: adrstrategichub@esrc.ukri.org.
Comment by CHRIS SMITH posted on
Thanks for reaching out. Great to see acknowledgement that Excel front/back end isn’t the way forward and the appreciation of the value of structured data in improving services. Public procurement is the means by which governments acquire goods, services & works and that enable services to citizens to be delivered. The UK and national governments have through their Open Government Partnership http://www.opengovpartnership.org action plans mapped out activities that aim to improve the quality and volume of public disclosure of procurement data by adopting principles and tools championed by the Open Contracting Partnership NGO http://www.open-contracting.org
However, a recent report http://www.instituteforgovernment.org.uk/publications/government-procurement by the Institute for Government (IfG) revealed that there was a significant problem with the availability of data to government about public contracts in the UK which in part is compounded by the lack of a procurement data standard across government. IfG recommended the adoption of the Open Contracting Data Standard by all UK Contracting Authorities. While the Open Contracting Data Standard has been adopted for the publication of data for Contracts Finder https://www.gov.uk/government/publications/open-contracting, this hasn’t been adopted by Contracting Authorities for their e-procurement systems resulting in a huge data gap which is adversely affecting government’s ability to obtain the best value for money and at the same time be transparent. The OCP has produced a helpful guide about what data should be collected by e-procurement systems https://www.open-contracting.org/2018/07/18/new-guidance-including-ocds-electronic-procurement-projects/ and published through APIs and I think it would be great for your team to engage with the Open Contracting Partnership team if you are interested in the multiple benefits an open source ready to use supported data standard for public procurement can offer.
Their contacts details are:
engage@open-contracting.org
Open Contracting Partnership, 641 S Street NW, 20001 Washington, D.C., USA
@opencontracting
Regards
Chris Smith
@bravemowzauk
Comment by Tom Ewing posted on
"While perhaps not the most glamorous, it’s important we continue to focus funding on things like getting away from Excel as the default front end (and often backend!) for our data."
Amen!!
Comment by Peter Jordan posted on
Absolutely yes - but it raises the bar for data skills for non-analysts.
Comment by Lisa posted on
What would you recommend as an improvement on Excel?
I would like to improve my data skills and it would be best to do that with the tools that will be used going forward. Where should I start?
Comment by Darren Barnes posted on
For anyone interested in the Government Statistical Services own statistical data revolution follow us here!
https://gss.civilservice.gov.uk/guidance/the-gss-data-project/
Comment by Prof Michael Chantler posted on
We (Heriot Watt University) have developed web tool designed to help open and joint organisation, by participants, of initial meetups and events s. We believe that the this greatly increases the usefulness/productivity of the meeting and feeling of real contribution and organisation by participants. In short, it's great for community building.
It has been used to organise over 240 meetings of up to 100 participants. It's free for academic use and we'd also be very happy if it was used for this type of activity. See http://www.well-sorted.org for a short video.
Procedure: The organiser defines a question to stimulate submission and scope ideas/issues/discussion points. Participants respond remotely. After a publicised deadline all participants can view all submitted ideas and individually organise them into groups. The tool then generates a report containing all of the ideas which are automatically ordered by similarity and grouped into clusters, as defined by the joint clusterings of participants. The report is normally sent to participants a few days before the meeting - and it usually forms the basis for the agenda and break out groups etc.
Comment by Steve Messer posted on
Changing how we create, use and share data in government is the next big opportunity, in my eyes, so I'm glad to hear you two saying the same! Though individual teams and departments might be getting better at using data to make decisions, there is a huge amount of infrastructural work which needs to be done – on standards, on sharing – to open up the possibility for government to innovate on its current offerings, at scale, nationally and internationally.
Having the skills to work with data is, for me, an important part of being a product manager. Understanding how data is collected, structured, manipulated, stored, analysed and shared allows me to interact with professionals from different disciplines: technical architects, developers, data scientists, performance analysts, open and linked data specialists, user researchers, and senior management. 'Data' is a word with an incredible wide scope; knowing what kinds of data are used and *how* needs a bit of breaking down.
Better service and design standards for data in government would also be hugely opportune. There's a movement of open data hobbyists doing sterling analysis work, and a set of guiding principles and self-certification to standards could help them create value for government directly. It's a patchy idea but one I explored in this post: https://medium.com/porism/democratising-civic-problem-solving-1a3c81a6d3b7
I need to think more about my answers to your 'What's next?' points but, in the meantime, sign me up to hear more!
Comment by Frankie Roberto posted on
Thanks for mentioning our work at the Race Disparity Unit! We’ve certainly worked across a few boundaries (particularly departmental ones). We’re not done yet though, as we are now actively looking at issues like standardisation and the potential to move from separate Excel/CSV files to a more structured format.
Comment by Bonnie Brown (CDIO) posted on
I'm aiming to work in a Data Analyst/ Data Scientist role within government, but am new to the field and don't have any previous experience.
Where should I start and how do I get into Data? What opportunities and resources are available? Where are the main Data Science roles based? Are there many in Newcastle?
Comment by Darren Barnes posted on
To add see here for more info https://gss.civilservice.gov.uk/guidance/the-gss-data-project/
Comment by Darren Barnes posted on
Kit and Peter we have been looking at this in government statistical service and have a working prototype in linked open statistical data. Get in touch to learn more. We are using the tools from the open government intelligence to help us get there with a few others.
Get in touch!
Comment by Henry Partridge, Trafford Data Lab posted on
This a very welcome post and we share your aspiration for a data revolution across government. We work in a local authority and have been considering many of the same issues and how to address them. Rather than data literacy we believe that a wider lack of digital literacy is the obstacle to the creation, use and sharing of data. Gaining an appreciation of the principles of the web would help as it is built on a number of open standards that facilitate the storing, reading and sharing of data. Any data revolution across government needs to adopt and promote these open standards.
We have recently been involved in the EU-funded OpenGovIntelligence project in collaboration with DWP Jobcentre Plus teams in Greater Manchester. The purpose of the project is to promote the creation and use of Linked Open Statistical Data to improve service delivery. The project tried to address many of the issues that you have identified and so could serve as a useful case study. In our pilot we developed applications co-creatively with end users, used open source web technologies to visualise data and published the code under an open licence to promote reproducibility. You can read more about our involvement in the project here: http://www.trafforddatalab.io/opengovintelligence/
Comment by Peter Jordan posted on
I'd agree that to get wider traction, there's a challenge to improve digital literacy - so that users can gain a better grasp of opportunities, but also to improve data literacy fundamentals - so that users can interpret data, challenge presentations, ask about provenance etc.
Comment by Andrew Newman posted on
Excellent blog post and I agree with most of it. How do we join in with your revolution?
On your point about no data equivalent to the service standard. Here at Defra we wrote a set of data principles that were designed to work with the service manual, be understandable by normal people and give our architects enough direction. They were widely praised externally but Im not sure they have landed internally (for all sorts of reasons) might be worth digging into them again...
Blog post: https://defradigital.blog.gov.uk/2017/04/04/defras-data-principles/
Principle Documentation: https://docs.google.com/document/d/1jTC7BCIipNDdrxi8CXVearQzANqL1AGUbvkXbOjnW5k/edit?usp=sharing
Comment by Peter Jordan posted on
Andrew - these are really helpful documents, which I hadn't come across before.