Considering Healthy Data before Big Data

Considering Healthy Data before Big Data

We’ve been giving some thought to the big data trend and the effect it’s had on the business intelligence space. The hype alone has pushed many big businesses to amass huge amounts of structured and unstructured data with the intent of gaining strategic insight and a competitive advantage. In some instances, the collection and analysis of big data is seen as integral to an organization’s genetic code – think behemoths like Walmart and Target.

The benefits are easy to ascertain – enterprise-wide insight, unprecedented dialogue with consumers and the ability to redevelop your products based on up-to-the-minute feedback and awarenesses.


Yet the pursuit of volume brings with it the pesky (and often overlooked) issue of data quality. Considering that the best of decisions are made by leveraging sound data, putting the issue of quality front and center may well turn out to be your company’s best asset.

Let’s touch on three essential reasons why this is so:

1. Excellent Data Quality = well-made decisions

Although data quality may never be as “sexy” a topic as Mobile BI or the cloud, it will always set the stage for skillful analysis and staying a few leaps ahead of the competition (who may not be as committed to the ongoing discipline.)

Achieving quality may involve a complex process of defining common definitions (naming conventions, what defines a customer? a prospect?); executing an initial data cleanse and maintaining order via ongoing data monitoring, ETL and other technologies.

Organizations may even consider hiring a Data Quality Manager and subsequent team to govern data processes, including migration,manipulation and analysis. The Data Quality Team must also ensure that reliable information is loaded into the data warehouse. Responsibilities may cross organizational silos, in that both IT and business users will need to step up and take responsibility for this initiative.

Keep in mind that many BI systems allow users to write back directly to the data source. If this is the case within your organization, it’s imperative that the user is authorized so as not to corrupt the system with erroneous data. In short, it takes strong communication and a team acting in tandem to ensure superior data quality.

Ultimately, success will depend upon trust. Users must believe that the informationthey are analyzing is “healthy,” timely and accurate in order to use it well.

Enforcing good data practices, particularly at the source system level will boost your credibility, reinforce sound analysis and save you volumes of time-consuming inefficiencies down the road.

2. Compliance Matters

In a post Enron world, it’s wise to consider that poor data quality may be leaving you out of compliance with the law. Although Sarbanes-Oxley (SOX) affects primarily public companies, businesses that undergo a merger, acquisition or IPO must also be in compliance, lest they face fines and potential lawsuits. Just another reason to keep data quality at “top of mind” status and perhaps allot a little extra in your data quality column during budget season.

3. It’s ineffectual to ignore

We won’t hit you with a bunch of catastrophic stories regarding the side effects of “unhealthy” data. We’d rather you consider the vanished productivity of workers tasked with fixing data problems, rather than performing their actual work; which leads to outsized costs in the end.

It pays to keep in mind that a task may cost approximately 10 times more when the data isn’t clean, compared to when it is. So go ahead and figure out the cost of a particular task and then multiply it by 10. That number should be enough to persuade you to dive in and make an investment in data quality, considering that the ROI is so easy to prove.

We hope we’ve convinced you to circle back to square one when mulling over the big data trend. It’s not that bigger isn’t better, but without data quality soundness, the hype is just an empty promise. Better to allocate your resources wisely concerning quality, than to face the music on a down note of lost time and effort.


Leave a Reply

Save Filter