Cleaning up your data is like cleaning up your diet; you know you should do it, but the doughnuts are too tempting!
It requires discipline, processes and accountability from leadership to prioritise. But it is often the most overlooked aspect of any data project. If your organisation has been ignoring it for years and you are now in the midst of a project you can’t finish, this article is just for you.
Here are three ways to overcome Data Quality issues in any analytics project.
There is a lot of unnecessary noise when the projects are going wrong.
You have to become a prioritisation ninja and understand what data truly matters. If your end output is required to conduct customer churn analysis, focus your efforts on prioritising data that will help in customer retention offerings. It’s simple, but in the real world, data is messy, and systems are disparate & undocumented.
Starting with the end will help you draw up a lineage all the way back to your problematic data and pinpoint exactly where your project resources need to put their efforts.
Real-Life Example
I was involved in a project to consolidate outdated client systems to identify the most up-to-date and accurate customer addresses for marketing campaigns. Nearly six months were wasted trying to clean customers’ address data in source systems, which ultimately was unnecessary. This is because the requirements were not translated accurately; you only needed their first line of address and postcode to target a customer. The rest of the data could be added using a reference dataset like Royal Mail’s Postcode Address File (PAF). Start with the end; what are you trying to achieve?
Most of the time, your data won’t need to be 100% complete and accurate.