Dark data. It sounds ominous, but it’s not malicious; it’s simply data that’s not getting seen. For many organizations, it’s causing problems but they don’t even know it.
While dark data does have the connotation of being highly secure information or having negative attributes, said Greg Milliken, vice president of marketing at M-Files, the company takes a broader view: it’s data that is important to a business that has become invisible because it can’t be found effectively.
Dark data has always been around to a degree, but as companies get bigger and more people are capturing information, knowledge ends up stored in separate systems and silos. “Important data doesn’t get found,” he said. “It’s getting to be a great problem as the amount of data increases.”
In some cases, said Milliken, users may find themselves recreating data that already exists somewhere else within the organization. And it’s not the data is dark to everyone: for example, an organization’s customer support department might have some information on a customer, while the sales team has other information. Each could benefit from the other’s knowledge; sales staff need to know that a customer is not happy and has made a great deal of support calls.
The democratization of technology has meant many different users are entering data into different discrete systems set up by the line of business, said Milliken. In the past, an enterprise content management (ECM) system would be set up and administered by the IT department, and the more sophisticated ones were difficult to use.
There’s no rule of thumb on what proportion of data in an organization is dark, said Milliken, and in some cases it’s okay for it to be dark because it’s just clutter, but he said the premise of M-Files that is there is a fundamental opportunity for all business and all vertical industries to improve how they handle information.
The opportunity is greater depending how a company was started and how it has grown, he said, and it may not be about the volume of data, but rather its criticality. The inability to find a small yet significant piece of information could have a negative impact on a business.
The ECM is largely considered mature, but for M-Files, everything is greenfield, said Milliken. The big ECM players may have all of the functions and features listed as part of their offering, but with the growth of cloud and mobile, the market is shifting. “A window has opened that needs to be revisited,” said Milliken. A lot of systems do what they do pretty good, but it’s often not very easy for the average user to connect the dots between different data, which inhibits effective adoption.
Opening up the world of dark data is reliant on connecting data in structured databases, such as CRM tools, to unstructured content throughout an organization. “The connection of these two is a clue to opening up the world of dark data,” said Milliken. The challenge with unstructured data is how users organize their files is completely arbitrary – their folder structure makes sense to them, but may not may not make sense to a co-worker. How information is organization is highly subjective based on the company and the individuals within it.
Unfortunately, there’s no ready clues to alert companies that they may have a dark data problem, although there are often indirect hints, said Milliken. An audit that took too long because information was difficult to track down or a bottleneck in a workflow. “Usually companies have some symptom — something else is awry in the company,” he said.
“Some processes aren’t working. Customer satisfaction is dropping. Audits are failing.”
