Today it’s become much too precarious for many government agencies to store mission-critical or sensitive data at a single location.
On the other hand, maintaining multiple data repositories at many different sites could trigger other challenges – mostly relating to the low speed, steep costs, and high risks associated with inter-site data transfer.
A recent federal-supported storage and information management (IM) initiative may prove to be the answer to this quandary.
Dubbed the Global Data Habitat (GDH), this project seeks to change the IM paradigm to enable both secure storage, as well as multi-site data collaboration.
Technology Partnerships Canada – an Industry Canada agency – has invested $7.67 in this initiative being spearheaded by Edmonton-based technology firm, Yotta Yotta Inc. at a total cost of $30.1 million. Venture capital firms are providing the remaining funds.
“Our goal is to create a new data habitat that doesn’t live in one place but adjusts to where information is being used,” said Wayne Karpoff, chief technology officer at Yotta Yotta. He said the new initiative combines heterogeneous storage and supercomputing to enable high performance data sharing. This, he said, will be accomplished by having “mirrored” copies of data at multiple locations. The GDH will be capable of managing two or more data centers separated by thousands of kilometers as a single, seamless location.
Global storage pool
According to Karpoff, using multiple data storage centers to provide scalability and fault tolerance is not enough. He said under the traditional “insular” model, data storage centers act like islands unto themselves. “Losing a data center could mean losing access to data associated with it. Besides collaboration between sites is slow and expensive as speed limitations impede data transfer across networks.”
The antidote, he said, a global storage pool that can be managed as a single entity. “We need a technology that links data centers together, allows site failures without loss of data access, and enables information to be shared between sites at local performance rates.
A tall order?
“Not at all,” says Karpoff, adding that Yotta Yotta’s NetStorager – a storage and data management product – allows organizations to accomplish many of the feats described above. He said NetStorager’s “intelligence layer” resides between data consumers (computers, servers, clusters) and physical heterogeneous storage systems developed by vendors such as EMC, IBM, Hitachi and others.
“Coherence operations within NetStorager ensure a consistent, worldwide data image,” Karpoff said.
Data fusion
Yotta Yotta’s new storage model, Karpoff says, can be used effectively by government departments, as well as by private enterprises across the board – from Fortune 500 companies to mid-sized firms.
However, he said, the most compelling applications are in areas such as defense, aerospace and security. It was the security aspect that Deputy Prime Minister Anne McLellan emphasized when announcing the government’s investment in the new technology. The project, McLellan suggested, is in line with the Canadian government efforts to “provide an integrated response to public safety and national security emergencies and threats.”
Karpoff explains how the Global Data Habitat can be a tremendous asset to Canadian intelligence or law enforcement agencies. “It’s absolutely crucial for these bodies to protect their data storage infrastructure so if a particular installation gets taken out the others can continue functioning.”
He said technology used in the GDH makes this all possible. “If one data center loses some or all of its systems, access automatically shifts to a surviving data center. If an entire center is lost, data access continues using other locations. When sites reappear, storage is automatically rebuilt with only data changes transmitted across the network. Many sites can be involved in both data replication and collaboration.”
This paradigm, he said, can be used to foster collaboration between agencies, by providing an effective way for the Canada Border Services Agency – for instance – to collaborate with U.S. outfits such as the CIA, FBI, and National Geo-Spatial-Intelligence Agency (NGA). “It also allows intelligence agencies to operate satellite data centers with real-time images of the same data. It’s a concept called Data Fusion.”
Breaking barriers
The ability to dramatically accelerate file retrieval over huge distances is another NetStorager feature that will be harnessed by GDH. For instance, in a trial involving the data transfer between Vancouver, Chicago and Ottawa, Yotta Yotta and its Canadian and U.S. partners reportedly broke the previous world record for bulk data transfer by a factor of 16. This capability, Karpoff said, is vital in matters relating to domestic security and infrastructure, where large amounts of data need to be moved quickly across networks.
In another test of data centers in Ottawa and Edmonton, sharing a single data image NetStorager delivered performance more than 10 times faster than single site performance. Yotta Yotta says, the test conducted by Mountain View, Calif.-based Silicon Graphics Inc., is another indication that with the right technology, long-distance shared storage really works!
P3 power
While it addresses the challenge of effective storage management, the Global Data Habitat initiative, is much more than a storage success story. According to Karpoff, it also has all the makings of a great private-public partnership (P3).
That’s something Industry Minister Lucienne Robillard also emphasized when announcing her ministry’s investment in this venture. The project, she said, is in line with the government’s commitment to “made-in-Canada solutions that promote public-private partnerships.”
According to Karpoff, GDH represents a win-win situation for all partners involved.
He said Technology Partnerships Canada’s contribution is not a grant but a recoupable investment, with the agency earning royalties on all new products based on this technology. “The agency is supporting a technology that’s of interest domestically, and is also likely to result in the creation of viable export products.”
Yotta Yotta is also partnering with other companies to test the new Global Data Habitat. These organizations may either incorporate the system in their own IT infrastructure or offer services based on the technology.
According to Karpoff the alliance between Yotta Yotta, the government, other technology firms and venture capital groups (such as Technocap, Optical Capital, Morgan Keegan, and Sapphire Capital) has all the ingredients of a profitable P3 initiative.
Such P3 ventures – experts believe – can serve as the antidote to the many challenges and demands of public services delivery. An recent IDC Canada study predicts government agencies will increasingly turn to P3 to deal with social, economic and demographic challenges to the delivery of public services. In fact IDC estimates federal P3 projects will double in the next four years – growing from Can$1 billion in 2004 to more than $2 billion in 2008. In the same period, the study says, “provincial government projects will nearly double rising to $3 billion,” while municipal government expenditure will experience the most spectacular growth – from $660 million to nearly $1.8 billion.
The study titled How to make public private partnerships successful in the Canadian government sector describes how government agencies can harness private sector alliances to enhance the level service they provide to the Canadian public.
