Miami-Dade County’s IT operations centre serves the largest metropolitan area in the state of Florida and is staffed around the clock. Not surprisingly, its managers have had to come up with innovative ways to balance the county’s IT requirements with the needs of the centre’s workers — especially when natural disasters such as hurricanes occur.
Labour Day weekend, when Hurricane Frances struck the state, was a prime example. In order to continue supporting vital IT services such as the county’s VAX-based 911 system and its police and fire radio network during the storm, the operations centre’s 650-person staff split into two groups working 12-hour shifts.
And to keep IT workers close to their loved ones during the hurricane, families were encouraged to camp out in the Miami data centre, said Ruben Lopez, the county’s chief technology officer.
“We try to be as accommodating as possible so people can be with their families, especially during the lockdown situation,” said CIO Judi Zito, referring to the IT department’s practice of not allowing anyone to enter or leave the data centre during emergencies. “I wouldn’t want to be here during a lockdown if my elderly parents were staying at my house,” she added. “I’d want to be with them.”
‘Juggling Act’
Zito’s flexible approach to balancing operational and staff needs during a disaster reflects the kind of strategies that many IT executives have developed during their tenures in the Sunshine State, which has been hit by two hurricanes in the past month and this week was bracing for a third — Hurricane Ivan — that at press time was due to make landfall in Alabama at approximately 1 a.m. EST on Thursday.
Staffing during hurricanes “is certainly a juggling act,” said David Gawaluck, CIO at Lakeland Regional Medical Center in Lakeland, Fla. To help support critical medical systems during the height of Frances, the medical centre had eight volunteers from its 65-person IT department stay at the 851-bed facility. “Fortunately, we have lots of beds here,” Gawaluck said wryly, adding that other IT staffers stayed at home so they would be fresh when needed.
“We worry about our people, because you need them to come back into work and they need to deal with their families and homes,” said Lew Temares, CIO at the University of Miami. The university kept 10 IT staffers at its central campus in Coral Gables over Labour Day weekend to monitor servers and communications systems during the storm, Temares said.
The school, which ranked second on the 2004 Computerworld (U.S.) list of Best Places to Work in IT, has a tight-knit staff of about 300 IT workers. One staffer drove to the home of a colleague who didn’t have water to deliver ice, according to Temares. Other workers who were in need of help received food and other supplies from a crisis management team that the IT department has set up.
Bacardi U.S.A. Inc., the U.S. importing, marketing and sales arm for wine and spirits producer Bacardi Ltd., took a different tack. It shut down its Miami and Jacksonville data centres on Sept. 2 and transferred data processing activities to an IT facility in Puerto Rico for five days, said Doug Watson, director of IT for the Americas.
SIDEBAR: IT Shops Prepare for Worst, Suffer Minimal Disruptions
Hurricane Frances dumped nearly 15 inches of rain on some parts of Florida. But IT managers interviewed this week said their data centres were spared major damage and mainly suffered power disruptions that led them to rely on diesel generators to keep systems running.
“We’re still paddling faster than the waters are rising,” said David Gawaluck, CIO at Lakeland Regional Medical Center. Soaking rains knocked out power to one of the hospital’s nursing units on Sept. 5, forcing the IT team to run the unit’s PCs and other equipment on generator power for four hours, Gawaluck said.
He added that two days before Frances arrived, the hospital sent a set of backup storage tapes to an Iron Mountain Inc. data center in Philadelphia as a precaution. As it turned out, Lakeland Regional’s data centre had electricity throughout the storm.
At Lockheed Martin Enterprise Information Systems, help desk calls that are normally split between facilities in Orlando and Denver were all routed to Denver over the Labour Day weekend, said Elaine Hinsdale, director of communications.
Company officials also decided to take three e-mail servers that support 7,000-plus Orlando-area workers off-line before Frances hit, Hinsdale said. In addition, they took down nine application servers across the state, including one in Lakeland that supports a significant amount of payroll and finance processing. Fortunately, Lockheed Martin’s Orlando data center didn’t suffer any physical damage, she said.
The University of Miami didn’t suffer any IT disruptions, but CIO Lew Temares said he and his staff learned a valuable lesson.
The university runs a server-based system from Diebold Inc. that lets students use combined identification and debit cards to pay for meals and gain access to various buildings. “We lucked out because our central generator didn’t go down,” Temares said. “But if it had, we could’ve lost access to the system for 18 hours.” The IT team now plans to use servers located off-campus to back up the system.
