Big data is useless without visual analytics

Big Data, with its vast data volumes, is largely useless without the data analytic and presentation functionality found in visual analytic tools. Think about how difficult it is to spot anomalies or trends in endless rows and columns of spreadsheet data. Visual analytic tools solve that data overload problem.

Big Data is an in-vogue topic in information technology just now. Many in the executive suite are starting to see that Big Data opens up transformative possibilities for products, services and markets. This understanding has sparked massive investments in software for visual analytics and business intelligence as well as related services that are often cloud-based. The sales of this software are set to grow significantly in the future. Wikibon, a community of business technology practitioners, projects the Big Data market will top $84 billion in 2026, attaining a 17 per cent compound annual growth rate (CAGR) between 2011 and 2026.

Big Data refers to the vast data volumes being produced by:

1. The Internet of Things (IoT), which includes the rapidly growing number of sensors in electronic devices such as smartphones, cars, appliances, industrial machinery, satellites and airplanes;
2. Everyone’s social media conversations, Web browsing and search history;
3. Business transactions as captured by point-of-sale (POS) terminals, loyalty programs and debit/credit cards;
4. Organizations of all types and sizes adding content to web sites;
5. Business applications and documents;
6. Reports and presentations created by governments, businesses, non-governmental organizations (NGOs).

Because of its large volume, Big Data is difficult to analyze meaningfully for business value. Let’s assess the tools many organizations routinely use to analyze their Big Data and see how successful each approach really is.

Visual analytics is just right for Big Data

Visual analytics is a component of business intelligence software that emphasizes:

1. Visualizations or charts as output;
2. A point-and-click graphical user interface for remarkably easy development.

Visualizations are valuable because they display a lot of data in an easy-to-understand visual format that works well for our visually-oriented minds.

Business intelligence software is a set of tools for the acquisition and transformation of raw data into meaningful and useful information for business analysis and improvement purposes.

Overall, business intelligence software packages are growing suites of software that:

1. Can process ginormous data volumes;
2. Integrate data from multiple, disparate data sources;
3. Offer a wide variety of visualization types that provide considerable formatting control and the ability to customize default values;
4. Include ad hoc query, charts, dashboards, proactive intelligence and alerts;
5. Model, forecast and project data;
6. Offer rich reporting services to build and publish interactive reports;
7. Are self-serve; meaning end-users can develop their own reports and visualizations;
8. Include data governance, security and role-based access to data.

However, business intelligence is undermined by a lack of:

1. Data quality such as wrong or missing values;
2. Secondary or supporting data;
3. Analysis manpower;
4. Organizational openness to new results.

Data mining is too complicated for most uses of Big Data

Data mining is the automatic or semi-automatic analysis of Big Data to extract previously unknown patterns that may be useful for business improvement. Data mining techniques can include artificial intelligence and machine learning.

Overall, data mining:

1. Focuses on sophisticated exploration of data;
2. Offers unexpected insights.

However, data mining:

1. Requires expensive specialists such as data scientists for effective operation;
2. Depends on traditional statistical methods that are unsuitable for vast amounts of Big Data;
3. Creates observations and related recommendations that may be difficult for the typical business management audience to relate to;
4. Produces unsupportable results when guided by preconceived notions.

Classical reporting is too ponderous for Big Data

Some organizations have made significant software tool and development investments to develop a rich library of reusable data analysis reports where the end-user can vary the data selection criteria dynamically. Leading examples are SAP Crystal Reports and Oracle Reports.

Overall, reusable reports:

1. Produce great-looking output for routine queries;
2. Deliver reliable, consistent results because the reports are carefully developed and tested by the IS department;
3. Can be produced efficiently when the data volume is modest and the number of data sources is low.

However, reusable reports:

1. Are not self-serve even though requirements are constantly changing;
2. Require software developer skills to develop and enhance. Therefore, their success is entirely dependent on IS department capacity and responsiveness;
3. Require software maintenance when the versions of the underlying applications are upgraded;
4. Don’t support exploration of the data;
5. May or may not provide basic graphing functionality;
6. Will choke when there are large volumes of data or the number of data sources grows;
7. Tend to proliferate over time as many versions with minor differences are created.

Excel is too restrictive for Big Data

We’ve all heard that Excel is the leading tool for data analytics. It’s widely and successfully used by smaller organizations with generally primitive applications, simple tools and modest data volumes. Excel is also widely used as a powerful personal productivity tool within many larger organizations where IS department responsiveness is a problem.

Overall, Excel is:

1. Cheap;
2. Self-serve;
3. Fast for developing quick analysis of modest amounts of data;
4. A marvelously flexible tool for manipulating data;
5. Incredibly easy to use to graph modest amounts of data.

However, Excel:

1. Severely limits the data volume it can successfully query;
2. Restricts the number of data sources it can access;
3. Uses syntax that makes programming and debugging difficult;
4. Too often delivers misleading or inconsistent results due to software defects;
5. Handles missing values inconsistently;
6. Produces primitive output;
7. Is not scalable to multiple end-users;
8. Is devoid of enterprise-level management features.

In summary, visual analytics is dramatically superior to the alternatives for achieving business value from Big Data.

What is your experience with achieving business value from the Big Data you’ve collected? Let us know in the comments section below.