In this column, John Webster from the Evaluator Group, an analyst firm that focuses on storage, makes the distinction between the problems and opportunties of storing lots of data and mining it. The former, he says, is nothing to be concerned about one way or the other– there hasn’t been much of a paradigm shift in storage, after all.
What’s important, he says, is mining and extracting information from that data, alluding to artificial intelligence. The great majority of “big data” is in unstructured form, and that is where the major innovations are going to come in.
I’m actually exploring the topic of unstructured data, artificial intelligence and insights myself in ComputerWorld Canada’s September digital edition. I focus on a distinction in analytics itself: the difference between making observations, as it were, from millions of pieces of data, versus drawing one inference from it all. They are both very different and require completely different types of infrastructure (inferences from massive amounts of data cannot generally be done on distributed platforms; this usually requires HPC technology).