It might not really be a plummeting avalanche, but to many IT managers the growing mountain of data they face is just as threatening.
Industry observers predict the volume of data – both structured and unstructured – will grow exponentially as companies find new uses for information, and regulators require more oversight.Small and medium businesses (SMBs) are hit particularly hard because, unlike their counterparts in larger enterprises, they often lack the resources to tackle this problem with a comprehensive strategy.Text
Companies with large stores of data often find it difficult to quickly locate what they need. For some, mining the valuable information contained in the data is a real challenge. For others, outdated technology, or cumbersome IT processes hamper the effective use of company data.
These problems may arise from specific circumstances, or they might be due to a deficit of technical skills, a failure of leadership, or simply a lack of money. Whatever the reason, data-saturated companies find themselves struggling with the growing demands of information management.
Small and medium businesses (SMB) are hit particularly hard because, unlike their counterparts in larger enterprises, they often lack the resources to tackle this problem with a comprehensive strategy.
What are the key elements of an effective data management strategy? And what can SMBs learn from the experience of large enterprises, some of which have been tackling these issues since the 1970s? This is the first in a series of articles that investigates the issue comprehensively.
The many facets of data management
“Data management,” says Wikipedia, the online encyclopedia, “comprises all the disciplines related to managing data as a valuable resource.”
The breadth of this definition hints at the range of IT disciplines included in data management: data modeling, database administration, data storage and warehousing, data mining, business intelligence, knowledge management, customer relationship management (CRM), data security, compliance and more.
Effective data management strategies encompass all or a great many of these facets. They cannot focus on one to the exclusion of others. Adding to this complexity is the constantly changing nature of data itself.
Surveys across multiple verticals indicate that the very nature of corporate data is undergoing a sea change. For instance, conventional business data categories – such as customer lists and parts inventories – are now being supplemented with digital images, voice messages, video clips and biometric profiles on computer databases.
With data itself becoming richer and more multifaceted, the load on processing and storage technologies is increasing exponentially, and so is demand for software tools to manage and extract useful information from the raw data.
Databases and BI tools
Databases are at the core of data management. Used to structure and store data so that they can be retrieved and manipulated quickly, databases have grown in power and complexity. Now database vendors, such as Oracle Corp. and Microsoft Corp., are streamlining products and simplifying user interfaces to meet the needs of SMBs.Using BI tools, companies can pull in data from a number of sources to produce cogent analyses of their market environment and competitive position in it. Text
However, even with the best databases, companies may need specialized tools to create custom reports or analyze trends and conditions indicated by their data. Digging even deeper for data gold, many companies are turning to business intelligence (BI) and customer relationship management (CRM) applications and methodologies.
Using BI tools, companies can pull in data from a number of sources to produce cogent analyses of their market environment and competitive position in it. BI applications typically allow for queries, and produce output, in language that is meaningful to decision makers.
A number of key players in BI applications are now touting their products’ ability to integrate data from multiple applications and funnel the queries and responses through a common interface, such as Microsoft Office, with which decision makers are comfortable.
CRM principles and practices go beyond technology to incorporate sales and customer service activities and certain aspects of marketing. CRM technology integrates and automates the various customer service processes within a company. The focus here is on automation of customer service processes and the gathering of information about customers and their preferences.
Many successful CRM applications, such as salesforce.com, have been Web-based, and this direction seems likely to continue.
When it comes to data analysis, vendors such as Oracle, Cognos, IBM and Hummingbird continue to offer new products that allow businesses to reach deep into their data stores to retrieve those hidden gems of information that can support fact-based decision making. Most of these companies are now focusing considerable resources on the SMB market.
Companies need strong skills in quantitative methods, along with leadership buy-in, to fully realize the benefits of an analytical approach.
Underlying all of these applications is the data itself, which needs to be safely and efficiently stored, yet readily available for manipulation when called upon.
Storage technology has recently been transformed by the availability of cheaper, faster and more robust disk drives. In particular, there is a shift away from tape backups to live disk mirroring and e-vaulting for faster and more reliable disaster recovery.
Lower disk prices also mean that SMBs can take advantage of recent advances in storage technologies to safeguard their data and recover quickly in the event of a disaster.
The long-running debate about the merits of storage area networks (SAN) as opposed to network attached storage (NAS) has been muted recently as companies learn that these technologies can co-exist, particularly where comprehensive practices such as information lifecycle management (ILM) are used to create a policy framework that focuses on the data rather than the technology.
Practices such as ILM are available to SMBs as well as large enterprises. In a sense, they help level the playing field by allowing smaller companies to compete — or at least interact more effectively — with bigger players. ILM is supported by many of the major vendors of data storage products, and SMBs may want to go with one of their products for the SMB market in order to leverage the skills and experience of the vendor.
Beyond technology
While technologies abound, it’s being recognized more and more that effective data management is not exclusively – or even principally – a technology issue. It goes far beyond speeds and feeds, and is intimately tied up with people and processes.
A recent report by IT research organization Butler Group, based in Hull, England emphasizes this fact. Titled ‘Data Quality & Integrity’, the report criticizes the typical response to data quality problems – which is to blame the IT department.
“Not only is this not helpful, it is not actually correct,” the report says. It goes on to say that data quality needs to be seen as a business – not a technology – problem. The solution, it says, begins with “implementing a proactive, ongoing data quality strategy.”
Unfortunately, most SMBs – and many larger ones – have not formalized such a strategy. In fact, Butler Group report dubs today’s levels of reporting, accounting, and publication of data as “lackadaisical”, an