Not too long ago, the IT function was for the most part still a separate supportive and enabling department that served business units with technologies. In order to evaluate and then fund their projects, CIOs were forced to predict a Return on Investment. They would measure IT success in dollars. ROI was the singular goal that got project managers sweating as they defended how their program would exceed a minimal threshold. ROI was typically 10% plus or minus, an amount below which projects died on the PowerPoint screen. Project prioritization and go/no-go decisions were based on ROI forecasts, which had better be convincing and better be right.
In those days metrics such as NPV (net present value) and PB (payback period) sometimes supplemented ROI. All else held equal, these metrics were used to anticipate project success. Post-project audits were attempted to figure out the payoff.
Today we recognize IT as an integral part of the business units. Technologies are inextricably embedded and are so fundamental to business processes that no one can imagine or tolerate cutbacks or downtime. Today IT success can’t be measured only in dollars – in ROI. There must be more.
ROI is a forecast of the dollar benefits expected after all costs, savings, revenues and taxes are accounted. But estimates, no matter how rigorously performed, are usually wrong. Hopefully the ROI numbers are not very far off. Regrettably, underestimating cost is as common as over-estimating revenue.
Post-project assessment of ROI is more problematic because all else is not held equal. Change happens – constantly! The impact of changes throughout the company and its marketplace invalidates meaningful post-project ROI determination. And don’t even mention scope creep. You simply can’t know.
Looking at how things get done
If ROI is flawed, then how should we measure IT projects? Measuring success from IT should focus on how processes are changed and most importantly on how those processes impact the organization. What any IT project should do is change how you conduct your business. IT should alter how things get done, how employees function and how customers are served.
To really determine the return on money spent we must measure what I call the “IOI”, the Impact on Investment – how the worker’s day improved and how the customer’s experience was enhanced. Efficiencies derive from workers. Revenues derive from customers. It is that simple.
Impacts tend to follow four main themes. Efficiency impacts look most like the old ROI efficiency targets of cheaper and faster, doing more with less. Effectiveness impacts look at how employees can be more effective, more flexible or more knowledgeable, as distinct from just faster. Opportunity impacts are those that create new possibilities that current technologies and information do not permit, a platform for yet more impact. Preparedness impacts are those that render the organization better equipped for “what-if” surprises.
To begin your IOI prediction, phrase the needs in those four themes and then canvass employees – both direct and indirect users – to determine how many expect to benefit from the application as outlined. Their anticipated impact will sort into the four themes. You will be able to determine how employees expect to benefit, how many of them will benefit, and to what degree (daily, weekly, occasionally). The IOI would turn that into a project goal, such as: X impacted weekly in year one.
Some IOI can be negative or costly. In that case the downside must be limited (e.g. not more than Y percent disaffected). With an ROI approach, negative impact can sometimes be overlooked. Things can still look rosy even though staff or customers are adversely affected. IOI enables negative factors to be openly managed and minimized – a much better tactic.
If, for example, 20% of Engineering employees expect beneficial impacts, then an IOI goal of 20% impacted in a stated time period could be set, depending on project rollout schedule. As you get more comfortable with the Impact themes, any of the four can be distinguished as sub-goals. At project wrap-up, canvass employees again to obtain a true post-project IOI level, complete with notes about how and why. If the IOI targets are not met then investigation and remedial actions are warranted. If the actual IOI exceeds the goal then a celebration is held. Further incremental innovations and broader rollouts typically compound the impact.
Compounded impact goals of 70% of engineering, 35% of marketing and 15% of customers are valid. The point is that real goals can be set and measured.
IOI is not difficult to determine and can actually be compared to the impact IOI goal. On the other hand, ROI is questionable to forecast and almost impossible to accurately measure afterwards.
Meeting IOI goals means an uplifted and more effective return was achieved. The key word is effective. IOI uplifted returns will bolster management’s confidence that the right IT decisions were made. That’s the kind of impact we’re all looking for.
QuickLink: 054331
–Greg Duffy is a Principal Consultant with Woodfield Consulting, focussing on the strategic value derived from IT and process investments. A graduate of the Rotman Executive MBA program, he is based in Oakville Ont. and can be reached atg.j.duffy@sympatico.ca