As Hurricane Ike bore down on Houston one Friday last September, the Continental Airlines’ flight operations centre, located on the 14th floor of a glass-sided downtown high rise, suddenly went dark. For the airline’s pilots and flight crews, however, business proceeded as usual.
Here’s why: At that same moment, 42 miles north of the city and some 60 feet underground — in a hardened Cold-War era bunker built by a paranoid millionaire oilman to survive a nuclear holocaust — Continental’s backup data centre took over. Throughout the ordeal — from Friday morning, as the storm approached, through Saturday, when winds above the Westland Bunker in Montgomery, Texas, gusted to 125 miles per hour, until Sunday evening, when operations resumed in Houston — the airline managed an 89 per cent on time rating for its global flight schedule.
Locating a backup data centre in an underground bunker may seem like overkill, even in a hurricane zone. But the facility met all of the airline’s requirements — including cost, says John Stelly, managing director of technology at Continental. The bunker, run by real estate partnership Montgomery Westland, has been converted into 33,000 square feet of rack-ready data centre space complete with air conditioning, redundant network and power sources, uninterruptible power supply systems and backup generators.
Continental leases 2,000 sq. feet underground and another 12,500 sq. feet of office space above ground, in a hardened building complete with 3-inch-thick bulletproof windows. The airline can house its entire operations staff of up to 125 people at the backup site.
After Hurricane Katrina, Continental began looking for a fallback data centre for use during hurricanes. Westland “was far enough away to be out of harm’s way but close enough for folks to drive to,” Stelly says. The blast-resistant facility is admittedly a bit much for even Continental’s backup needs, but the four-feet-thick walls and high security entrance are nice extras, Stelly says.
Also, connectivity options at the Westland facility were a plus. The network and power feeds for the bunker were sourced from areas well away from Houston, while pricing was competitive with above-ground co-location facilities.
Rise of the underground
With a renewed focus on data centre outsourcing and space in high availability facilities in short supply, investors such as Montgomery Westland have snapped up and renovated abandoned mines and military bunkers in the hopes of cashing in.
Since 2007, for example, Cavern Technologies has operated a data centre 125 feet below ground in an abandoned limestone mine. The mined out area underground, which covers 3 million sq. feet, is 15 minutes outside of Kansas City, Mo. Unlike other mines, the Cavern facility was created with the idea of reuse in mind, so floor space isn’t irregularly shaped like other underground facilities can be, says president John Clune. The area’s relatively low electricity costs, at 3.5 cents per kilowatt-hour, help to make operating costs lower than those in other parts of the country, he adds.
Another facility, The Bunker, is a decommissioned, 50,000 sq. foot Royal Air Force bunker that operated until the 1990s. The facility is inside a hill near Dover, England and it now hosts data centres 100 feet below ground. “People get a picture of a hole in the ground. That’s not the case. It’s a state-of-the-art data centre,” says Paul Lightfoot, director of managed services for The Bunker. Clients range from businesses running mission-critical Web applications to a financial services firm that runs online trading systems. “We do everything from basic square footage to fully maintained systems,” he says.
Iron Mountain is among the oldest and best known providers of underground storage and data centre space. Known for storing everything from backup tapes to old movie reels in The Underground, its repurposed limestone mine in rural Pennsylvania, the company has seen its electronic storage and leased data centre space business increase while its traditional paper record storage business has slowed. “It is now the fastest growing component of our business,” says vice president Charles Doughty. In addition to leasing rack-ready space, the company offers data centre design, hosting and management services.
But while interest is up, the number of actual customers leasing space in its underground data centres remains small. Iron Mountain counts five operating data centres in its underground facility, including its own. But with 60,000 square feet of available data centre space and another 145 acres undeveloped in the facility, Iron Mountain has plenty of room for more.
Underground data centre facilities fall into two categories: Abandoned mines, like Iron Mountain’s, and decommissioned military bunkers such as InfoBunker, a subterranean facility just outside of Des Moines, Iowa. InfoBunker leases data centre space to organizations ranging from a local telephone company to government agencies about which Jeff Daniels, a vice president at the company, says he can’t talk.
On the demand side, an increase in extreme weather events, heightened concerns about security since 9/11 and the need to provide higher levels of security to comply with regulatory requirements have made these spaces more attractive to some organizations. Underground facilities offer security and structural protections that would be cost prohibitive to build from scratch.
Meanwhile, the recession and credit crunch have made it harder to get funding to build new data centres, and organizations have become more accepting of the idea of using co-location facilities to house mission-critical data centre operations. “Demand for computer space is stronger than I’ve ever seen it [and] the supply is so small, so inadequate,” says Peter Gross, vice president and general manager at HP Critical Facilities.
Basic co-location space isn’t the problem, says Jon Bolen, chief technology officer at Westec Intelligent Surveillance. The surveillance-monitoring service, based in Des Moines Iowa, serves clients such as McDonald’s Corp. and Zales Corp., and recently completed a search for a hardened facility for its own backup data centre. During this search, Bolen saw a general lack of high-end infrastructure, of enterprise-class data centre space. “If you need space that is as good or better than the space you would build there’s a shortage of places you can go.”
This shortage has given the underground facilities an opening to pull in larger data centre clients. Cavern says it is negotiating with Fortune 500 clients looking to lease spaces of 30,000 to 100,000 square feet. However, most clients are smaller organizations that don’t require so much space; more typical would be a hospital that leases 1,500 sq. feet.
One might assume that IT organizations would have to pay a premium for bunker space. After all, the cost of building such a structure is high, and special venting and air-flow systems are required. But IT executives say they’ve driven deals where the total cost of ownership is competitive with above-ground facilities. Because they’re repurposing existing space that the government or a mine operator paid to build, providers say they don’t have to pass on the original construction costs for the structures and can afford to be cost competitive.
Consider the cons
Before deciding to go underground, IT executives need to identify potential limitations, experts say. Even things as simple as ceiling height can be a challenge. Continental’s data centre space in the Westland bunker has 10-foot ceilings, and putting full-height racks on top of an 18-inch raised floor was a tight fit. “We had to come up with a design to allow us to use full-height racks while providing sufficient airflow,” Stelly says.
Another concern: While computer systems may be protected in a bunker, critical infrastructure needed during a disaster, such as generators, fuel tanks and air conditioning cooling towers, may be above ground. That could be a problem if the catastrophe you need to worry about is a tornado, warns Westec’s Bolen.
Bolen recounts how one company claimed that its hardened facility could withstand a direct hit from an F3 (158 to 206 mph) tornado. But the air conditioning and generators were outside. “When an F3 hits, those generator and HVAC units are going to come off their pads,” he says.
Westec ended up taking space at InfoBunker, about 45 miles away from its offices, Bolen says. The 65,000 square foot Cold War command bunker, designed to withstand a 20-megaton nuclear blast, maintains all infrastructure, including generators, fuel and cooling equipment, 50 feet underground.
Another consideration is that these underground facilities tend to be in rural, out-of-the-way locations. The facilities may be too far away from a company’s primary data centre, and finding local lodging for staff in a disaster situation may be difficult. Continental had to find office space and lodging accommodations for more than 100 operations staff during Hurricane Ike. Fortunately, Montgomery Westland had hardened above-ground office space as well as access to local lodging.
Underground facilities do have a few other advantages. The limestone floors at The Underground have a virtually unlimited load rating, while the walls maintain a constant temperature of about 55 degrees and act like a heat sink for some of the waste heat that comes off data centre equipment. The limestone walls absorb 1.5 BTUs per hour per sq. foot of wall space, Doughty says.
Cool stuff
The green aspect of going underground is what attracted Marriott International. It wanted to move from an outsourced “cold site” disaster recovery service to managing its own hot site backup data centre. Management wanted a hardened, secure facility in a location that was within a day’s drive from Marriott’s Bethesda, Md., headquarters. And it wanted to make sure the facility followed the company’s focus on environmentally friendly best practices, according to Dan Blanchard, Marriott’s vice president of enterprise operations.
Last year, the hospitality business completed the build-out of a 9,000 sq. foot remote backup data centre at The Underground. Blanchard says that although the extreme level of security, including armed guards, exceeded his requirements, the idea of reusing an old mine rather than breaking new ground appealed to Marriott. “It’s a definition of recycling to use the space that was a mine and convert that fairly inexpensively to its next use, which for us is a data centre.”
Energy efficiency also factored into Marriott’s decision, Blanchard says. While Marriott’s data centre uses a traditional chiller as its primary cooling system, the backup is a prototype free cooling system. That prototype, designed by Iron Mountain, uses an air-to-air heat exchanger, drawing 55-degree air from the 1,000 acres of unused space within the mine. “The air is the exact temperature of what you would bring in with mechanical cooling,” Doughty says. Iron Mountain also is experimenting with a system that would pull cool water from an underground lake within the mine.
An abandoned mine may conjure up images of damp walls and dripping ceilings — but that’s not the case here. “You have pumps and a lot of protective devices,” says HP’s Gross, and all of the facilities claim that dampness is not a problem. Doughty says The Underground is naturally dry due to its location and the type of limestone above the mine.
Air quality also is good, he says. The air in the Iron Mountain facility is relatively clean and non-condensing, he says. “As soon as you put heat to it moves away from the dew point,” and that makes it a good choice for cooling, he says.
Blanchard says the new Recovery and Development Centre, which is used for software development until needed in an emergency, costs half as much as he previously spent on power. Some of that is attributable to relatively low cost of power in Pennsylvania (5.5 cents per kWh). The rest comes from efficiencies of design and the characteristics of the underground environment.
Gross cautions, however, that cooling efficiency gains specific to the location are probably not all that significant. A well-designed data centre today can cut power consumption in half by using new energy efficient equipment that can run at higher operating temperatures, by optimizing airflow designs to allow intake air temperatures to rise as high as 85 degrees and still keep equipment within operating temperature limits, and by picking a location in a colder climate, where water- or air-side economizers can be used to take advantage of cool outside air as weather permits.
Security, Gross says, is the primary benefit of using an underground facility to host a primary or secondary data centre. But for most of his clients, the ability to get people to the backup data centre in a hurry, connectivity options, and finding a facility that meets budget are priorities. Underground facilities usually don’t beat out above-ground sites in his clients’ evaluations, he says.
Still, Continental and Marriott are among a small number of enterprise operations using underground facilities. Rakesh Kumar, an analyst with Gartner Inc., says he is unaware of any Gartner client that is currently leasing space in one. The primary benefit of such sites, he says, is that they are designed to be highly resilient — often to military specifications. That’s important for some government data centres. “But for most commercial enterprises, it probably will not be such a major requirement.” IT executives considering underground data centre space should check into expansion capability, energy efficiency and how electricity use is metered, he says.