Data Democratisation and the Trouble with Rolling Averages
21/09/2017
![data-screen-1](https://blogs.cranfield.ac.uk/wp-content/uploads/2017/09/data-screen-1-1.jpg)
I was watching a webinar the other day with various luminaries adding weight to a subtle sales pitch. Two of them were talking about future trends – one claiming data democratisation (Google it!) was spreading like wild-fire and was to be encouraged, while the other spoke about it being a myth. I’m with the latter – and much more in favour of information democratisation. Letting every person and their dog have access to any data so they can interpret it any which way they can – in my view that’s a recipe for much activity, confusion and wasted effort…
Let’s take just one of many examples – the application of moving average charts to data. It is often used to look for trends, cutting out the amount of “noise” (uppy-downy movements in data over time) in the data. Investors use it for example to cut out the ups and downs in trading to see where a certain stock is generally heading – and good luck to them! Those trained in statistics may use these kinds of charts to smooth out seasonality so they can see any signals where data is breaking out of that seasonal pattern.
So what is the effect when we look at some of these charts?
Here’s a 12-month moving average chart of some data (out of context to demonstrate the point):
So we’ve got a roughly constant set of results from July 2013 to June 2014 – nothing much going on.
But putting this in context we see a different picture – one that we might want to take action on:
So, in context we see there is a strong seasonal pattern, with high volumes in July and August and low volumes in February! The Moving Average Chart above would look like the same picture if the underlying data were flat rather than seasonal. So the Moving Average loses information about what is actually going on! It does not help in taking corrective action. The extended-SPC chart (showing the seasonal pattern) allows officers the opportunity to focus their efforts when Incident volume is high in July and August in order to have maximum impact.
Here’s another example:
So now we have a signal in February 2014 – something must be special about February 2014. “Find out what happened in February 2014” rattles down from the corridors of power.
But wait up! When we look at the data in context, we see a very different picture:
So actually, there’s nothing special about February 2014 at all! February is just part of a run of low results since June 2013 (when actually earlier corrective action was taken by the force to reduce shoplifting). The February 2014 signal in the Moving Average Chart above is actually as a result of the previous high values for months up to February 2014 dropping out of the 12-month moving average calculation! So the Moving Average Chart can be misleading taken out of context – which they usually are!
Lessons Learned:
- Don’t use Moving Average Charts out of context
- Don’t use Moving Average Charts to look for signals of corrective action impact nor for opportunities to apply corrective action
- Recognise that by processing any set of raw data potentially removes information from that data.
- Apply Moving Average Charts only when appropriate.
Or alternatively, you could standardise on the one Dilbert chart above – it might be just as useful!
Categories & Tags:
Leave a comment on this post:
You might also like…
Keren Tuv: My Cranfield experience studying Renewable Energy
Hello, my name is Keren, I am from London, UK, and I am studying Renewable Energy MSc. My journey to discovering Cranfield University began when I first decided to return to academia to pursue ...
3D Metal Manufacturing in space: A look into the future
David Rico Sierra, Research Fellow in Additive Manufacturing, was recently involved in an exciting project to manufacture parts using 3D printers in space. Here he reflects on his time working with Airbus in Toulouse… ...
A Legacy of Courage: From India to Britain, Three Generations Find Their Home
My story begins with my grandfather, who plucked up the courage to travel aboard at the age of 22 and start a new life in the UK. I don’t think he would have thought that ...
Cranfield to JLR: mastering mechatronics for a dream career
My name is Jerin Tom, and in 2023 I graduated from Cranfield with an MSc in Automotive Mechatronics. Originally from India, I've always been fascinated by the world of automobiles. Why Cranfield and the ...
Bringing the vision of advanced air mobility closer to reality
Experts at Cranfield University led by Professor Antonios Tsourdos, Head of the Autonomous and Cyber-Physical Systems Centre, are part of the Air Mobility Ecosystem Consortium (AMEC), which aims to demonstrate the commercial and operational ...
Using grey literature in your research: A short guide
As you research and write your thesis, you might come across, or be looking for, ‘grey literature’. This is quite simply material that is either unpublished, or published but not in a commercial form. Types ...