With IT budgets tight, CIOs are being asked to tally up ROI figures for every sizable project these days. Not surprisingly, consulting firms ranging from Gartner Inc. to two-man start-ups — often founded by Gartner alumni — have stepped in with a full complement of ROI metrics, methodologies, guidelines and benchmarking services to cash in on the opportunity. We’ve sifted through some of the offerings.
Business Value Index
The Hackett Group recently revamped and renamed its IT benchmarking methodology. An intriguing characteristic of its Business Value Index is that it offers clients the ability to compare themselves against other enterprises. Hackett has benchmarked 2,000 IT organizations, including those at General Electric Co. and Citigroup Inc., on their alignment with corporate strategy, ability to partner with suppliers, and level of technology integration, according to Bruce Barlag, the firm’s president and a former Gartner ROI analyst. Barlag adds that client companies are able to examine the practices that help top-quartile corporations succeed.
From Hackett’s extensive database, you can learn, for example, that the average IT organization has 11.5 full-time IT employees per 1,000 company employees, while at high-performing companies, the number is less than half that — 5.2 IT workers per 1,000 company employees.
The other major Business Value Index differentiator is its continuous focus; many customers, including The New York Times Co. and Alcoa Inc., use their initial results as baselines for ongoing measurement. “Most companies benchmark every two or three years, but clients tell us they want more of an ongoing process,” Barlag says.
Hackett says a Business Value Index analysis can take as little as two weeks. The firm’s Web-enabled software gathers, scrubs and validates data; consultants collaborate with clients throughout the process and then deliver a final report.
Strength: Hackett’s large existing database can be used to compare IT practices and business value on a continuous basis.
Weaknesses: May not offer the hard numbers you’re looking for; moreover, the index is new and untested.
Information Economics
Tom Bugnitz, president of The Beta Group, stresses that although his Information Economics methodology encompasses metrics, they are simply a means to an end — that end being “a holistic view of all IT investments and an understanding of the linkage between IT spending and business results that lets management make decisions.”
The Beta Group is another consultancy that includes Gartner alumni among its principals. The key differentiator of the firm’s approach to IT ROI assessments is its focus on communication rather than data analysis.
The firm’s methodology includes four steps. First, client companies list and prioritize their strategic business endeavors. As part of this process, The Beta Group forces clients to use standard language across all departments and projects.
Second, the IT organization is asked to catalog its technology investments. Again, standard language and metrics are used. With this information, The Beta Group builds what it calls a “project book.” At this stage, says Bob Rouse, a company principal, it often becomes apparent that business managers know shockingly little about where the IT group is spending its money.
Next, a scorecard committee, composed of business, financial and IT executives, develops a relative valuation for each IT project. According to Bugnitz, this development process is the core of Information Economics, because it forces executives at the client’s company to understand and analyze their IT program in detail.
“People on this committee must learn whatever they need to know about every project,” Bugnitz says. “We provide a framework for disciplined thinking about business communication.”
Finally, the company creates a scorecard, which is a list of proposed IT expenditures ranked according to their business value. Once a Beta Group client that was faced with a $50 million IT project backlog ran out of money halfway down its list — and simply eliminated the rest, with no ill effects. “The CEO said, ‘If the process works, it works,’ ” says Bugnitz. A typical Beta Group engagement lasts six to 12 weeks.
Strength: Clearly ties IT projects to overall strategy, offering a good big-picture perspective.
Weaknesses: Requires a major effort from the scorecard committee, and the results of the exercise are only as strong as that group’s commitment.
IT Performance Management Scorecard
Mike Bitterman, a principal at IT Performance Management Group (ITPMG), is yet another Gartner refugee — he worked in benchmarking at Gartner’s measurement division. There, he says, “CIOs would tell us, ‘Something’s broken, and we don’t know how to fix it.’ And benchmarks don’t tell you how to fix anything.” So Bitterman moved on to co-found ITPMG and developed the IT Performance Management Scorecard.
In an ITPMG engagement, the client company establishes critical success factors associated with the health of its IT organization. These factors vary depending on how IT is regarded — as a utility, as a “demand” organization that delivers applications when requested, or as an “enabling” group that truly helps plan corporate strategy. (Brutal honesty is needed here: “Everybody wants to be an ‘enabling’ IT group, but very few are,” Bitterman says.)
With critical success factors and metrics determined, ITPMG’s scorecard software, which the firm installs on clients’ servers, helps companies develop eight to 15 key performance indicators. The goals: unambiguous metrics, a translation of typical IT jargon into business-value terms. (“Somebody still measures CPU cycles and all that,” Bitterman says. “But our software automates your ability to turn that into useful information.”)
Developing a scorecard for one department of a company takes six to eight weeks.
Strength: Provides the hard ROI numbers that business executives want to see.
Weaknesses: When it comes to translating age-old IT measurements into information that’s useful for business people, “there’s still a lot of missionary work to be done,” Bitterman says. To fully exploit the IT scorecard, a business must be ready and willing to confront the historical divide between IT and other organizations.
Total Economic Impact
Giga Information Group Inc.’s methodology essentially expands on traditional cost analysis by adding benefits and flexibility to the mix. Chip Gliedman, a Giga research fellow, says it’s important to factor in flexibility because investments in infrastructure, excess storage capacity or network bandwidth look like red ink in a cost analysis — but offer flexibility that can pay off in time.
Where benefits of an IT project are concerned, Giga, like nearly all its competitors, forces clients past generalities such as “boosts productivity.” Gliedman says IT executives tend to slip into a discussion of features but that Giga’s methodology keeps them on track. “We don’t talk features, but rather business benefits, and we always add ‘as measured by’ and a metric,” he says.
Typically, a Total Economic Impact assessment begins with identification of an IT project’s goals. IT then determines technology costs. Next, affected business units decide what benefits they stand to gain.
Then flexibility and risk are factored in. “An IBM server will cost more than one from Joe’s House of Servers” but is a safer purchase, Gliedman says, adding, “Risk-adjustment lets you decide how much of a discount you require from Joe.”
Finally, the results of the assessment are communicated to all concerned parties, and metrics for measuring the project’s success are determined.
Giga offers Total Economic Impact assessments as a consulting service or will train businesses to perform their own assessments. A full engagement takes four weeks or more, depending on scope.
Strength: Takes flexibility and risk — major factors in real-world decision-making — into consideration.
Weakness: In some companies, those additional considerations may dilute the ROI message.
Total Value of Opportunity
Total Value of Opportunity (TVO) is a methodology based on the Gartner Business Performance Framework, the company’s set of business metrics. Gartner analyst Audrey Apfel claims that’s an important differentiator: “Generic ROI cases tend to go right from geekspeak to financial numbers, and that’s too big a leap,” she says.
TVO is available as a Web-based software package, with full training-and-consulting contracts optional. To get started, a business uses Gartner’s business metrics to determine what opportunities are available and translates them into standard business terms such as on-time delivery, bill rate, market share and sales close index. One of TVO’s strengths is that this translation — which is where many efforts founder — is laid out in excruciating, step-by-step detail, eliminating ambiguity.
Another TVO differentiator is the software’s diagnostic capability, which weighs such qualitative factors as the company’s IT project history. “Lots of clients tell us, ‘We have a great ROI process, we run decisions past our accountants — and then we go on our gut feeling,’ ” says Apfel. “We tried to make that gut feeling visible.”
Strength: Specific instructions can’t be fudged, so results should be accurate.
Weakness: May add a level of complexity that meat-and-potatoes executives won’t welcome.
