Brian – Asking good questions is just a starting point in your analysis. While your BI tools / analyst can do something about say duplicate records or missing values, you can’t expect the tool to beautify flawed data and expect miracles in your results. One cannot make assumptions at every stage and produce a flawless analysis no matter how big an expert he is.

Olivier – thanks for bringing out these valuable points. I’ve so often experienced your point of losing a lot of time redoing things within a BI tool because something changed in the underlying data! And yes, null can be a big problem in BI tools – esp when your data source is something like Oracle tables which have NULL.

Stephen, I agree about your point that we need to agree upon data definitions and sources. And definitely there needs to be better data governance in firms – the IT and business will have to work together to achieve this. Ultimately, you don’t want to lose out on some critical points in your analysis because of flawed data.

For me the quality is fondamental. Of course, you can grow flowers with dirt, but not a whole garden…

The problem with BI is once a data is in the pipe, it is difficult to pull it out. If you forget something, or make some mistake, you will need a lot of time and effort to identify and get rid of the problem.
You can be a master analyst, but you can’t fill in the gap if data is missing.
You can guess though, but you always will have a doubt and so your analysis would be tainted. How can then be confident in the analysis and the result if we doubt of the data ? And this will be more certain if the result of the analysis is not what we expected…
Then of course, you will need good analysis tools and skills and know the business you’re working on.
I remember a dashboard that I made some years ago about the whole process of buying a car. In Uk for exemple it’s common to test a car before buying it. So the document had been made in function of that caracteristic. When applying that scheme for Belgium and Greece, an alert had been raised and data was considered bad. Well it is not common in Belgium or Greece to test a car before buying it. So the result of the analysis would have been wrong even with correct data because the business analyst didn’t know about that fact.

You can have a ferrari, that doesn’t make you a schumacher…

There are several cause to bad data. I will point 2 of them. First in my experience, a lot of problem comes from excel sheets sources. Excel is not a database and has no real constraints and so permits a lot of dirt. You have no control over data put into excel sheets.
Second, poorly designed applications. In order to win programmation time, often we loose some control, the initialisation of data is not good (null is not a great value for BI, is it a fault data or an intend value ? How can we know when we get the data at the end of the chain ?),…

Need to agree on data definitions and meaning, need to agree on data sources and need good analytics. Plus, when filtering the data, like most analytics do, need to proveide the context so people understand if they are looking at a view of data and is that view consistent.

Net is people need to trust the data and that starts with good data quality and some data governance around the data assets.

@Ron – I definitely agree about your view that there needs to be a good mix of both data quality and good analysis. I did feel Mr. Morey’s article was a bit one-sided as well when I first read it. But I can’t help but feel that companies these days invest heavily in hiring top analysts and in utilizing the best technologies and in the bargain lose focus on the underlying data! To put it plain it is like ‘garbage in’ – ‘garbage out’. To get the best out of your analysts and tools, there needs to be more emphasis shown in keeping your data clean and consistent.

The issue with striving for better quality is that it’s a lot like expensive preventative medication. If it works in 97% of the cases but then there’s a miss, people will complain that “See? I spent all that money and I STILL have the problem!” or even if it’s 100% successful most people will say “See? I was fine after all and I didn’t need to spend all that money.”
Same goes with good analysis. If you do a good job someone would look at that and say “See? Didn’t need good analysis because the data is so good.” Just like in his scenario where he points out the stats are only valuable against the entire league. Well, really that’s a finding made by someone doing data analysis (even if it’s Mr. Morey).

The sword cuts both ways. Accordingly it should be employed both ways.

Good data is like music and analysis the musician. A good musician can play a poorly written piece wonderfully but they will never make it seem well written. Likewise a bad musician can take a wonderfully written piece and make it sound terrible. In the end it doesn’t really matter how well the music is written if the musician can’t read music.