The needs of transaction-oriented businesses — whether they’re in telecommunications, finance, or retail — are poorly met by most off-the-shelf software. The shortcomings of mainstream database management systems are reflected in posting delays and data lifetime limitations. For customers, it means hearing, “That isn’t in our computer yet,” or “That data isn’t in our systems anymore,” when they call. Customers don’t want to hear those responses any more than management wants to hear that the analysis it needs can only be refreshed each Friday at noon. It harkens back to the days of punch cards and batch processing.
Yet such delays are considered unavoidable by most IT organizations. Warehouses exist because it’s just not smart to query against a rapidly changing database. But many types of businesses need real-time access to data. They also require scanning for the patterns and trends that identify fraud or opportunity.
I have seen vertical solutions designed to enable real-time database access and analysis. I was not aware of more flexible, horizontal approaches to real-time challenges until I spent some time with Dave Wilson, Hewlett-Packard’s director of marketing for ZLE (Zero Latency Enterprise). The company’s ZLE project combines best-of-breed technologies (most of them right off the shelf) to create a fast, distributed database that you can hit with complex queries while transactions are coming in. Essentially, HP’s ZLE delivers the benefits of a data warehouse without imposing its limitations.
Wilson demonstrated ZLE using a readily understood model: a busy e-commerce site. A long row of rack-mounted servers split the duty of providing ZLE services, emulating shopaholic customers and running real-time queries. ZLE sets itself up as the operational hub for data, messaging, and rules-based analysis. Much of HP’s magic is invested in teaching a standard database manager (in the demo, Oracle) to respond to queries in real time. HP cracked the problem of keeping query responses synced with incoming data. ZLE also prioritizes activities so the system will make sure the most important transactions get the shortest turnaround.
Two components of HP’s ZLE really intrigue me: the rules engine and the messaging hub. Wilson’s e-commerce demo relied on a dynamic set of rules that altered the shopper’s browsing experience based on things such as buying history, click tracking, and demographics. Think of personalization but on a grander scale than a packaged solution can deliver.
An interesting part of the demonstration involved fraud detection. In the e-commerce demo, Wilson simulated a set of merchandise returns that triggered an internal fraud alert. The status of the customer’s account was changed so that if one more questionable return were submitted, customer service would be notified.
I wouldn’t have been impressed by Wilson’s ZLE demo if it lived only within HP’s headquarters. At the time, HP had sold it into some 30 accounts. If the setup is daunting, it’s only because there are so many pieces, and the companies that can afford ZLE have incredible problems to solve. But as ZLE and concepts like it are deployed, real-time features will start creeping into mainstream database managers. Dave Wilson predicts it’s the end of the data warehouse. I hope he’s right.
