MeasureCamp is a free, informal ‘unconference’ for digital analysts, held on a Saturday in London twice a year. Its objective is ‘the transfer of knowledge’, and everyone is encouraged to contribute, either by running a session or by sharing ideas and experiences. We’ll cover some of our highlights and favourite tips from the latest event.
It was standing room only in many of the sessions, with about 300 people attending from all over Europe. Three performance analysts from GDS managed to get a ticket. The first challenge was choosing which sessions to go to, with 8 rooms simultaneously hosting presentations, workshops and discussions on a range of analytics topics.
Some of the recurring themes of most interest to us were data quality (and quantity), measurement frameworks and how to measure content performance, understanding the user experience, and potential uses of machine learning for analytics.
Top tips
1. To avoid ‘vanity metrics’ or collecting data just because you can, focus on what you’re actually going to do with it. A simple performance framework could consist of:
- what’s the objective?
- what’s happening to achieve it?
- how will you measure it?
2. Overall averages can be misleading, so segment by different device types and browser widths to spot issues that are only affecting a certain group of users.
3. Don’t use sessions with pages in a Google Analytics report - use unique pageviews instead. See the Misunderstood Metrics series of blog posts from Analytics Edge.
4. Examine your most and least ‘navigated-to’ pages, by calculating the number of unique pageviews minus entrances. Which pages are users choosing not to go to within the site?
5. Look at failed site searches, for example search terms with no results. (On GOV.UK we track the number of results in a custom dimension.) Which ones are phrases that users are using to find things but you’re not mentioning in your content? Which ones are about content or services that users want but you don’t (yet) offer?
6. Do you tend to discover that technical changes have been made without you knowing? Set up analytics alerts to notify you about any important changes in your data. You could also use an automated tool to regularly audit your analytics tags.
Other interesting sessions
The ‘Data: People: Ideas’ session considered the perennial challenge of moving beyond reporting to getting teams to act on insight. Building dashboards is often not enough. It’s important to add context, otherwise the data can be garbage. Building collaboration into a dashboarding platform would encourage others to have conversations and ask questions about the data.
Nicolas Malo presented an interesting model for planning and collecting customer experience metrics, grouping them into 4 types:
- behavioural - how users behaved on your digital property, tracked with digital analytics
- attitudinal - what people thought of your service or site, typically collected through surveys and feedback, which can be structured or unstructured
- technical - how well the service or site is performing technically, for example availability, page load, broken links
- operational - what contact users had with your organisation, through call centres, webchat, etc
Béate Vervaecke led a discussion on the various different navigation and flow reports available in Google Analytics, and why their metrics are inconsistent. One of the likely reasons is different sampling levels in different reports, while a suggested cause of same-page self-referrals was refreshes from http to https. It was reassuring to hear from other analysts having similar frustrations.
Kristoffer Ewald ran a session on deduplication vs privacy and the implications of approaches to joining up sessions by the same user. Much of the discussion revolved around where the line between acceptable and intrusive tracking should be drawn. In particular, the group considered the difference between technologies like third-party cookies (used with consent), device fingerprinting, and predictive algorithms which join or dedupe based on patterns of behaviour (no consent). It was good to see that the group took the ethical questions very seriously and questioned the business model which offers free use of a service with tracking and selling of data, or paid use with more privacy. This was summed up in a very pithy comment from Ivan Bager: “Privacy is a right, not a luxury for the rich.”
Yali Sassoon’s session on data warehouse and digital analytics best practices began with a hands-up to see who in the room was already using or planning to use data warehousing. Plenty of people were. That in itself prompted some discussion of why this is becoming such an interesting subject. Many people spoke about ‘joining up lots of different views of the customer’, while somehow managing to avoid the MeasureCamp Swear Jar’s ‘360 degree view’. There were also mentions of mirroring Google Analytics or Adobe Analytics data elsewhere to ensure ownership of the data in the long term.
Measuring attention: Ed Brocklebank began with a quick summary of the flaws in how Google Analytics calculates time on page and session duration, then went on to show us how to use timer scripts to increment a custom metric every 5 seconds while a user is active on the page. Ed went into very specific detail about his approach - how he’d arrived at the 5-second time, the merits of custom metrics and events for the timing information, and he even demonstrated the script in action (on conference wifi - brave!).
More information
Presentation slides and blog posts will be collected on the MeasureCamp website soon. See other people’s comments and photos from Twitter in the Storify roundup.
The first MeasureCamp Newcastle is on 21 May, with free tickets released from 18 March. The next MeasureCamp London will be in September, and there are other MeasureCamps around the world.
Tara, Tim and Peter are in the performance analysis community at GDS.
6 comments
Comment by Nicolas Malo posted on
Many thanks for attending my workshop and for the summary! I've just uploaded the slides here: http://fr.slideshare.net/nicolasmalo/measurecamp-customer-experience-metrics-workshop
Comment by Charles posted on
Glad to see some of my tips made it into this post. Slides going up this weekend
Comment by Tara Stockford posted on
Thanks Charles. Here are the slides: http://www.slideshare.net/charlesmeaden/tracking-gaps-in-the-user-experience
Comment by Sean Burton posted on
Measurecamp will be coming to Newcastle-upon-Tyne on Saturday 21st May 2016 - more details available at http://newcastle.measurecamp.org
Comment by John Hope posted on
It took me several paragraphs to work out what you were talking about. I'm sorry, but this is a blog about data and it is not enough to say you are doing "analytics", "performance analysis", "digital analysis". This is the subject of 'analysing google analytics data' or 'web use stats' or maybe there is another name. Put it in your title.
Comment by Tara Stockford posted on
Hi John, sorry if this wasn't clear enough. We changed the remit of the 'Data at GDS' blog a few months ago to focus on sharing practical tips about our work with Google Analytics and other data tools: https://dataingovernment.blog.gov.uk/2015/12/08/changing-the-data-at-gds-blog/
Peter compared various digital analytics job titles and roles when we decided to call our team 'performance analysts': https://dataingovernment.blog.gov.uk/2014/12/01/time-for-a-rebrand-product-analysts-become-performance-analysts/
There's a separate 'Data in government' blog for the cross-government data programme: https://data.blog.gov.uk/