Big data is all the rage in IT and in business publications. Supposedly producing big business results from big data requires big upfront investments. But is that really true? Do you need to make big investments upfront to achieve big business results? Is there no way to start small and simple with modest investments before making big data investments that may be risky and may not produce the expected big business results?
Big business results
Big business results typically mean using the data that your organization collects plus external data to accelerate achievement of your business plan. For example, ambient temperature trending up or down will change the mix of products you will sell. Better understanding how the mix varies by temperature will help you minimize out-of-stock or overstock conditions to maximize sales. To quantify the impact, you can correlate temperature trends against changes in unit sales by day and by product during the past few years. You can then use the correlations to improve your sales forecast by product to maximize sales.
This example and many others can produce big business results without big investments because the technology required to produce your improved sales forecast is within the grasp of even mid-sized organizations with small IS capabilities and budgets.
Are big upfront investments necessary?
In the big data literature, there’s frequently the implication that you have to sift through huge haystacks of data to find a few actionable needles of insights. That approach to big data requires big upfront investments in specialized software, considerable data management, and skilled staff to operate the solution.
But is that really true? What big business results can you achieve using only the software and the databases you are operating today?
Leave data in place
Big data purists point out that transforming data and integrating data from multiple data sources improves query performance and illuminates actionable insights that are not obvious.
However, many organizations have only millions of rows of data, rather than billions. Also, many organizations operate only a small number of data sources. For these organizations the benefits of simplicity resulting from leaving the data where it is, far outweigh the cost and complexity that results from transforming and integrating it. Similarly, with lower row counts, the performance improvements achievable through transforming and integrating data are not material.
Leaving data in place avoids considerable software development and IS operating costs while accepting a modest reduction in analytical capability.
Use existing tools
Big data purists point out that using specialized software ensures data integrity, improves developer productivity and controls software maintenance cost over time.
However, many organizations have accumulated considerable experience using PL SQL or Transact-SQL for data manipulation and Excel for query and graphing. Often that’s enough to achieve big business results from the available data.
Some of you may bristle at including Excel in the small and simple solution because you’re frustrated with the limitations of Excel. However, Excel is incredibly powerful and a cheap place to start.
Using existing tools leverages available experience while avoiding big investments in specialized software and associated developers while giving up only a little in analytical functionality.
Understand the path forward
Once you’ve achieved a few big business results with this low investment approach, you’ll have identified further opportunities to achieve more results. If some of these new opportunities require you to make some investments, go for it. At this point, you’ll have a solid understanding of the benefits case and where you want to make the big data investments.
You’re now on a cheaper, clearer, lower risk path than if you had made the big upfront investments.
Can you share any examples of big business results with this low investment approach?
