Some organizations seem to think that big data can only produce big business benefits if it’s delivered by way of big projects that require big investments and include big risks. Nothing could be further from the truth.
Here’s a look at how to respond to the frequently-discussed issues that create avoidable anxiety and cause big data success to elude you.
Too much concern about Hadoop or a data warehouse
The current trade literature contains a raging debate about whether to use Hadoop or a data warehouse or both for managing big data. How about neither?
All your data resides somewhere. Perhaps the fastest and cheapest path to business benefits is to simply access data where it sits. Most of the available software can integrate data on the fly as queries require it. Few organizations own so much data that this simple model bogs down.
Too much focus on analytics software
The software market place offers analytics software with spectacular features and exciting software developer productivity. But do you really need to spend a bunch of money to license more software and train your staff to use it before you can deliver business benefits?
Most of the software packages you operate and are paying for include ad hoc reporting, query and graphing capabilities. Excel can produce considerable amounts of analytics and you’re already paying for it. Perhaps you should exploit this software to the extent possible before you acquire more software.
Too much fretting about visualization
Visualizations looks cool and can truly deliver important insights that may have been missed with more primitive software. However, visualizations can also overwhelm your end-users with a kaleidoscope of colors and too many over-posted time series of data.
So start the big data ball rolling with small projects that access a limited number of datasets for a small, select end-user work group to deliver comparatively simple visualizations with irrefutable business benefits.
Too much frustration about IS lack of responsiveness
Once again the IS department is being pummeled for alleged lack of responsiveness in failing to deliver. This time it’s about big data with analytics and visualization. Perhaps your organization is asking for the impossible after having just cut the IS budget yet again?
A better approach is to offer a richer self-serve environment for analytics and visualization. This approach improves responsiveness, accelerates delivery of business benefits and reduces the pressure on the IS department. Check out my related article: How advanced BI tools are delivering self-service analytics.
Too in love with big data
Have you hired too many data scientists who are in love with big data? Are they running amok spending money on grandiose projects that have produced only disappointing results to date? Data scientists are great but often the missing secret ingredient to achieve business benefits is an intimate understanding of how your business actually functions and its data resources.
Instead of a grandiose project consider a small project that integrates internal data with a modest amount of external data that will challenge a data scientist and produce business benefits by addressing a customer service challenge.
Too much governance and privacy standards
Has your organization implemented too much governance oversight and too many controls related to privacy standards? Are these processes gumming up the works and undermining your end-user access to big data?
I’m all in favor of reasonable governance and maintaining compliance. However, when a focus on these topics takes priority over extracting business benefits from the data you collect, then your organization has lost its balance.
To right the balance consider that most analytics is about identifying trends in aggregated data that doesn’t violate privacy standards for data about individuals.
Can you share any experiences about big data, analytics and visualization success?