Online trading broker Qtrade Investor depends on a rock solid network to serve the customers of more than 175 financial institutions across Canada. Billed as Canada’s leading independent online brokerage, the Vancouver-based Qtrade provides quick, easy access to a range of investing resources, including real-time quotes, charts, news and research. The company this year went from being in the top 10 to number four on The Globe and Mail’s listing of online brokerages in Canada, helped by a recent Web site upgrade that added real-time account updates and online bond trading.
Since 2000, Qtrade has counted on Fusepoint Managed Services for securing its mission-critical business applications. “We have a strong server platform, strong application server, fast Internet connection, and fast CPUs,” says Aldo Pippo, Qtrade’s chief technology officer. “Our people who are working with clients need to be able to click and go, click and go and I need to know that it’s always going to work; it’s never going to be down. Through Fusepoint, we can actually supply that.”
While Pippo and his team handle their own proprietary applications, Fusepoint provides them with physical co-location services, including an Internet feed that Pippo says is fully redundant with three different connections. Qtrade counts on Fusepoint for firewall services, intrusion detection, load balancing for redundancy and performance, VPN services back to the main site, backup and monitoring. Fusepoint provisions the hardware, loads the operating systems, loads and runs the Oracle database and iPlanet. It also responds 24/7 to system alerts and alarms.
Pippo says relying on Fusepoint to this extent frees his company to put resources into products and customer service, i.e., Qtrade’s core strengths. “We do not have to worry about the client calling up and saying, ‘Why is your Web site so slow? Why is it down all the time?’ We don’t get those calls.” Relying so heavily on another company could have its challenges. Pippo says the key is to keep each other informed. “They can’t do their jobs if they don’t know what we’re doing.”
In turn, Qtrade’s staffers listen to their Fusepoint colleagues. Pippo cites as an example Fusepoint’s warning that an application was generating large log files. As it was not a high volume application, he did not consider it a high priority issue. Sure enough, as soon as the product started taking off, the log files escalated and slowed down both the system and the backup. He enjoys being able to count on an outside firm to flag a potential problem before it becomes one.
Moving in-house
MidAmerica Bank went the opposite route. This company moved from managed services to bringing everything in-house.
MidAmerica Bank is a subsidiary of MAF Bancorp and one of the largest financial institutions in the U.S. midwest. Customers depend on the bank to rapidly process loans and provide reliable access to account information and 24-hour ATM transactions.
Until last year, MidAmerica relied on an external company’s disaster recovery site. This covered the bank’s core financial application only and required that backup tapes, which could be 24 hours old, be physically transported to the recovery facility hundreds of kilometres away from its Chicago headquarters. Restoring the bank’s primary system would take 24 hours in theory, but more likely 36 to 48 hours in reality. Fortunately it was never required. Another party stored backup tapes.
Paul Stonchus, MidAmerica’s first vice-president and data centre manager, recalls the company assessed three options: upgrade with the existing firm, keep some disaster recovery in-house or keep all disaster recovery in-house. “While it is hard to cost-justify returns, we did see that the cost of being all in-house was no more than any of the other options and we had the advantage of having control,” he reports.
In fact, he claims to have lowered costs and boosted customer service by aligning the IT infrastructure to address the changing value of information over time. A multi-layer information lifecycle management system manages and protects data across various storage tiers in line with business continuity requirements. With its own disaster recovery site, the bank constantly backs up data offsite.
Initially, MidAmerica used EMC Global Services to design and implement seemingly all of EMC Corp.’s multi-tiered networked storage and information management software for comprehensive storage for business continuity, protection and recovery. The full out investing in replication technology and infrastructure gave “the most bang for the buck,” says Stonchus.
Since the five-month implementation was completed last December, MidAmerica has been replicating these systems, in order of importance to the bank: the core financial system, loan tracking, ATM operations, cheque imaging, and reports and document management.
The cost of top tier storage with the fastest recovery was minimized by moving data as it ages down to levels of less accessible storage, freeing up the more costly high-end space for more frequently accessed data. For example, once a cheque clears or a loan closes and becomes “fixed content,” MidAmerica uses content-addressed storage (CAS) to archive these images for up to seven years and comply with federal retention regulations. Through CAS alone, MidAmerica has cut SAN storage requirements for imaging in half.
Stonchus also reports the firm has gone from 3 terabytes (TB) of direct-attached storage in a single data centre to 26TB of multi-tier networked storage across two mirrored sites without hiring additional storage administrators. MidAmerica’s entire storage infrastructure is managed by just two people because the system is so self-sustaining.
He cautions that it is important to automate as much as possible and have very good documentation. “If you have all these bells and whistles and replication in place and nobody knows how to use it, what benefit are you going to get from it? Make sure that it gets implemented so that your first level personnel can perform easily during day-to-day operations as well as in a disaster.”
Separate processes
Dave Donelan, senior director, Industry and Compliance Solutions at data management system manufacturer EMC, offers another tip: treat backup and recovery for compliance and for business continuity as two processes. “Backing up your business to a single point in time and recovering your business for business continuity is a business operation, versus archiving data for a specific number of years for compliance or corporate governance,” he explains. “Part of what companies wrestle with is having a firm and well-communicated policy and keeping the data they are required to keep by the regulators (e.g. Sarbanes-Oxley and Basel II) — and discarding the other information according to policy.”
EMC encourages customers to ensure their current backup/recovery and business continuity policies cover more than just structured data in one or two applications. Donelan estimates that unstructured data such as text in a word processing document and semi-structured data like e-mail, which is growing at an incredible rate, already account for 75 to 80 per cent of data. He preaches the need for financial institutions to follow a well-documented business continuity policy established with the institutions’ risk management officers’ input on how much downtime is acceptable for the various data types and ages.
And, be sure to do regular testing. “The more complex your environment, the more you want to make sure you capture all the data in your test,” he cautions.
Companies face media totally different from equipment that was available 10 years ago. He points out that with the Check 21 Act, cheques are being scanned at a greater level and, along with the tremendously growing volume of e-mail, provide both compliance and business continuity storage challenges.
Beyond the data centre
Companies in the financial services industry typically focus on protecting the data within the confines of the four walls of their data centre, yet they typically have remote or branch offices, adds Eran Farajun, executive vice-president of Asigra, a Toronto-based firm specializing in distributed data backup and recovery solutions for network computing.
Farajun argues that protecting data at multiple offices is costly and very difficult to do with traditional tape backup and restore solutions, which typically require “someone not very technical at the remote site to switch tapes and get data off-site. So, accountability is very difficult with traditional tape backup and restore products.”
He says single-site business continuity products are very differently architected than multi-site software like Asigra’s Televaulting for Enterprises. The latter removes the manual processes with software installed at each remote site to automatically perform a WAN-optimized backup that compresses the data, transmits it with AES 256-bit key military grade encryption into a customer’s central data centre where it can be managed by storage professionals. Rather than installing and charging for agents on every targeted machine, the software is licensed, based on the total compressed capacity being protected across all the sites. He claims this costs 50 to 70 per cent less than traditional backup and restore software.
Farajun cites a divergence of solutions and products coming to market as a general trend in the business continuity and security arena. The downside is greater complexity from products more specialized for specific types of business problems, he says. “On the up side, companies have more options and they can buy better products designed specifically for their requirements.”
However, he advises that this makes it vital to “make sure you clearly convey your business requirements to whoever you’re buying technology from so you buy a product that fits your business requirements.” Farajun sees another trend: greater outsourcing of security and business continuity. He cites Toronto-based OnX Enterprise Solutions Inc. as one of the companies Asigra works with. OnX delivers a managed backup and recovery service for customers.
“The financial sector is an educated, demanding buyer of managed hosting services,” adds Fusepoint CEO Robert Offley. He has noticed a trend toward utility computing where companies buy IT and related services on a pay per usage basis. Buying on demand enables a company to “match costs more to your revenue and your growth, and be far more agile in the marketplace.
“Our vision is that companies will buy software licences as a service more than just buying managed hosting,” he continues. “Managed services will evolve into utility computing where software is a service. People will be looking to buy licences to be hosted and managed, doing the patch management and the security, rather than just buying the licence.”
Already, Offley sees Fusepoint as a critical part of a customer’s IT infrastructure. Certainly it appears so for customer Qtrade for whom Fusepoint recently migrated the core infrastructure to a 10-server UNIX-based environment running iPlanet and an Oracle database. “Fusepoint is like an extension to the IT department,” agrees Pippo.
— Maclean, freelance writer/editor, covers a wide range of IT applications. She is based in Guelph, Ont., and can be reached at www.sumac.net.
