In some organizations, testing the performance of applications used to run the business or to render a service to customers can take the form of a centralized function that conveys the message that testing is not just a mere afterthought, said an industry analyst.
A “Centre of Excellence” (CoE), a phrase coined by Theresa Lanowitz, founder of Minden, Nevada-based research firm Voke Inc. and former Gartner lead analyst, describes a function that centralizes previously scattered skills “to tear down the internal silos” between units like quality assurance and development, for instance.
“Application performance is one of these things that has been largely ignored for a long, long time,” said Lanowitz. Despite the fact that applications are complex and drive the business, organizations still neglect to have business requirements or service level agreements associated with the testing, she said.
Lanowitz’s research report entitled Performance Centers of Excellence, to be released on Feb. 9, is based on interviews with almost a hundred global companies typically in the 20-50,000 employee range, that either have a CoE or are in the process of building one.
Large companies reap the benefits of economies of scale of a centralized resource, said Lanowitz, but smaller companies that build a CoE from the outset will be able to scale as the company or product and service portfolio grows.
Lanowitz listed five advantages that companies with CoEs reaped:
1.Increased productivity through a centralized function that preserves investment in performance solutions;
2.Decreasing costs through centralizing resources and skills;
3.Increasing institutional knowledge that can be easily transferred;
4.Increasing quality and reduction or elimination of performance defects prior to production; and,
5.Elevating the status of performance testing as a key component in the “triangle of quality, cost and schedule.”
But while tackling performance testing internally through a CoE is beneficial to some organizations, Lanowitz noted that it’s not necessarily the only way to go. “It’s really up to the organization,” she said, as some perceive their IT function staffed with project managers who handle contracts with outsourcers, whereas others prefer to have specific skill sets in-house.
Most organizations interviewed by Lanowitz preferred a CoE, but those that outsourced wanted the benefits of 24/7 insight into the testing process.
A CoE is typically not a headcount intensive activity, with 37 per cent of surveyed companies having only 5-10 direct reports. In peak times, contractors will often be brought in to help, said Lanowitz.
CoEs will report to the chief information officer or the architecture group. The latter arrangement, she said, gives a “very good vision of how they want their applications to actually perform” consider that testing is about uncovering architectural flaws.
Scott Barber, chief technologist with Palm Bay, Florida-based software testing consultancy PerfTestPlus Inc., thinks that building a team like a CoE, with skills and experience that fit more closely, is critical “because all of us are going to have different perspectives to be able to help with different pieces of the puzzle.”
However, performance testers often suffer from a misperception that hinders their role in the organization, said Barber. Product managers can be reluctant to integrate performance testers into the development process from inception, often wondering how testing can be relevant before anything is even built. But, Barber continued, testers can assist developers and architects validate theories and assumptions as they’re creating the applications.
“Part of the problem is that we call it testing,” said Barber, referring to the misunderstood roles. “And the fact of the matter is, there is that whole notion of when the developers are done, someone then should check it out and make sure it does what it’s supposed to do.”
There are some factors necessary for a successful CoE, said Lanowitz, with the biggest factor being buy-in from the IT and line of business management. “It’s a big, big thing.”
The CoE should also use a standardized tool, have a focus on customer service to ensure service level agreements are met, and be able to prioritize applications into production.
Also, a CoE must be staffed with the appropriate skills in software engineering, architecture, mathematics, modeling, usage of performance testing tools, business scope, and communication skills.
Barber thinks communication skills are critical to implementing a CoE because, very often, it’s doomed to becoming an “isolated, remote consulting arm” where the function becomes “handing applications and findings back-and-forth” without conversation.
But Barber said that’s changing, albeit slowly. “Maybe there is an uptick in a number of teams… [but others] are blissfully unaware and not evolving at all.”
Lanowitz acknowledged that a few years ago, IT and the line of business didn’t communicate much, however, one surprise her research unveiled was that organizations were realizing the importance of an ongoing dialogue in business language with the business.
