“Big data” is hot. We know it is because a) every industry analyst firm put it in their top five trends for 2014 and b) as of this morning there were 826 million articles mentioning it on Google.
Almost every CIO and director of IT in the world is considering the possibilities of leveraging big data, aside from the fact that there isn’t a commonly accepted definition of the term.
But a lead data analyst at TD Bank says it’s one of the most over-used terms in technology.
“Storing vast amounts of data in an efficient manner does not benefit the enterprise if you’re not generating insights that drive marketing or business strategy, ” Leila Lavaee told a Toronto big data conference on Wednesday. “It’s not about big data, but right data.”
Broadly, big data can be defined as large datasets that can’t be handled by conventional analytic tools. Working at a bank that has huge data sets of customer and Web site user information, Lavaee has some experience with it.
But volume of data isn’t important she argued. Organizations have to first define business questions that need answering, then go after the data needed.
Everyone knows the amount of information organizations can tap is exploding, she said, but so is the “noise” — irrelevant data. It’s easy to capture data, but some thought has to be given to using it strategically.
“Big data should drive your understanding of your customer,” she insisted — otherwise what’s the point of brining all that data in? So the business questions that need answering include what’s happening, when did it happen, why, what is likely to happen, what should be done about it.
“It’s a matter of what data will aid our business drivers, and then ensuring the right data is collected to be analyzed. Lots of businesses now work in an agile matter, and the ones that are moving really fast collect (data), analyze and respond, then loop back to the question to make sure what they are not falling away from the initial business question. They are constantly assessing what are the data points they need to capture the support of the business goals and strategy.
“There needs to be more connection between IT and the business side,” she added. I think that’s what the majority of organizations are not doing a good job of —- technology is doing whatever they want, business has a different requirement, and at the end of the day what’s implemented is something totally different.”
Another conference speaker, Robert Wong, Toronto Hydro’s executive vice president and chief risk officer, illustrated the challenges of big data. The utility, which distributes electricity to 725,000 customers.
The utility takes a holistic approach to collecting data, he said, meaning it captures information through hundreds of thousands of home smart meters, metered transformers and switches on its distribution system, its call centre and Web site.
It stores some 6 TB of data now, which is growing at 1 TB a year.
Some of that is put to good use, he said – for example, residential customers can go online and get information about their own use of electricity and how it compares to a similar-sized home.
But, he complained, “we’re just inundated with so much data today we have to figure out what subset of data before you go and analyze it.”
“Business really needs to understand what they’re chasing — what are their business goals and objectives, and what information they really need.”
In an interview he expanded on that. “We are collecting this wealth of data but it’s very complicated. There are hidden benefits in (the data) that are not obvious at first.” The more staff exploits the data the more they will think what else can be done with it, he said.
“It’s up to the business (side) to grow into it and understand the treasure trove of useful stuff there we can exploit and add benefit to the company, to the customers and our shareholders. But it takes time and maturity before we get there and fully exploit it.”
But he acknowledged that Toronto Hydro isn’t yet leveraging the most out of big data. “You would think it would come normally through departmental business plans, but it is a bit of a struggle. And I think part of it is they don’t know what’s possible.”
He agreed the Lavaee that there has to be a collaborative dialogue between IT and business units. But, he added, “you can bring a horse to water but you can’t make it drink.”
“Business and IT have to work hand to hand,” Lavaee said, “because at the end of the day IT has to consolidate infrastructure and help take analysts to next level to help them analyze the data. Without IT consolidating that infrastructure, analysts cannot do the job they were supposed to do.”
