During the mid-1990s, Robert Kaplan and Dick Norton developed the ideas for a Balanced Scorecard (BSC) approach to measuring long-term organizational performance. Until then, stakeholders relied mainly on financial statements as the chief measure. The BSC approach postulated that a combination of other high-level metrics, combined with financial performance, would be more indicative of a longer-term outlook for continued organizational success. In other words, additional measures, dealing with strategic plans, people management, customer satisfaction and relationships, would provide a more balanced outlook for sustained performance.
The BSC concept is equally applicable to measuring the success of Information Technology service providers and can be much more effective than traditional IT measures. This article explores how a BSC for Information Technology would help both IT executives and stakeholders gauge success and identify areas for improvement.
THE CASE FOR AN I.T. BSC
Metrics are not new to the world of IT service provision. For many years infrastructure measures, such as availability and response time, along with user support measures, such as time to resolve problems and call abandonment rates, have been used. On the applications delivery and support side, on-time/on-budget project performance and user satisfaction with implemented systems have been evaluated. Some organizations have embraced function points as a key measure for productivity trends as well as an effective means for estimating future development efforts. Again, some organizations have specific service level agreements (SLAs) for critical services and rigorously measure actual to target performance on these SLAs.
While very worthwhile, these measures fail to answer such questions as:
• Has IT delivered the right business applications and services to support the important aspects of the business?
• Are the IT professionals well led and well motivated?
• Are the service levels “fit for business” and affordable?
• Are things getting better or worse?
The application of the BSC approach can answer these questions and provide a more reasonable understanding of IT success along with potential areas for improvement.
THE I.T. BSC MODEL
The accompanying “BSC Dashboard” illustration depicts six “dials” representing a balanced scorecard for IT performance and service delivery. These dials represent what is important to an organization in the leadership for and delivery of these services. Individual organizations and corporate priorities may vary, thus changing the composition and priorities of these dials, but under this model the “dashboard” represents the following:
DIAL 1: IT Leadership
Increasingly, organizations are realizing that IT has to be strongly integrated with many of the critical business processes of the organization’s value chain. In some industries it is viewed as an enabler for strategic and competitive advantage.
This dial tests the ability of the CIO and his/her management team in aligning IT strategies, policies, resource deployment and plans in the best interests of the organization. It also tests the CIO’s ability to play an important role in the development of the organization’s business strategy with fellow executives, thus ensuring that enabling technologies are integrated with business processes, supply channels and delivery channels where practical. Through this, it ensures that investment decision-making for IT resource acquisition and deployment is in the best interests of the total organization.
Some of the specific measures to assist in positioning this dial include:
• Return on investment from approved business cases;
• Quality of IT’s long-range plans and their integration with organizational plans;
• The correlation between IT resource allocations and important core business processes.
Often, these evaluations may be qualitative, and extensive executive input is required to position this dial.
DIAL 2: Project/Capital Management
The delivery of new or enhanced business applications and services is typically performed through a project identified as part of the capital plan for IT. Sound project management practices and excellent project managers are required to develop quality plans, control project progress to plan, and manage costs to allocated budgets; not to mention to deliver a project which effectively addresses business requirements.
Some of the specific measures to support the positioning of this dial would include:
• “As built” product specifications vs. business requirements (often determined through post-implementation reviews);
• Actual/plan project schedule performance;
• Actual/budget project cost performance;
• Ongoing productivity trends (as measured by function point analysis or equivalent).
DIAL 3: Operations Management
Once new or enhanced business applications and services are delivered through the project environment, they need to be operated, supported and maintained. This dial measures the effectiveness of “fit for business” operational processes, procedures and controls, as well as the effectiveness of cost management within the day-to-day operational environment. Also, longer-term measures should determine whether performance and efficiency are trending in the right direction through continuous improvement programs.
Most large installations have used operational measures designed to indicate availability and responsiveness of various operational services, as well as the professionalism of the individuals involved in day-to-day delivery. What is often lacking, however, is the trend analysis required to monitor performance and productivity improvements. Such a trend analysis is needed to adjust this dial on an ongoing basis.
DIAL 4: Client Satisfaction
This dial is often measured subjectively, but is nonetheless as important as the others. As a “business within a business”, IT service providers have a customer constituency, and measuring their satisfaction (as well as the trends in satisfaction levels) is critical. Through these measures, IS responsiveness, professionalism, the effectiveness of communications, as well as client engagement in IT projects and operations can be gauged.
Typically, client satisfaction surveys are used as the measurement instrument for this category. These can be formal surveys periodically administered and/or follow-up questionnaires from the Help Desk to a subset of those who had called during the last reporting period. Dedication of quality client personnel to development projects is also a key indicator. And finally, “Noise Level” cannot be overlooked as reinforcement for client satisfaction (or dissatisfaction).
DIAL 5: People Management
Key to the long-term success of any leadership and services organization is the attraction, retention and development of good people. This dial is geared to indicate the sustained performance, productivity, career development and professional motivation of those providing IT services and leadership.
Some measures that will position this dial include:
• Voluntary departures — a negative indicator, particularly if high performers are leaving;
• Involuntary departures — potentially a positive indicator if poor performers are being managed out of the business;
• Promotion rates;
• Transfer rates (both within the IT area and transfers in and out of IT);
• Training investments;
• Distribution between contract professionals and employees.
DIAL 6: Asset Management
The final dial in the model deals with the management of assets entrusted to IT leaders. Certainly, there is a budget allocation to manage. Also, however, longer-term fiscal forecasts (and their accuracy) are key.
In addition, certain corporate assets have been entrusted to IT stewardship, such as hardware and software investments, as well as the data resource. All three must be managed from the perspective of security. Hardware and software should remain technically current but affordable, with IT maximizing its buying leverage.
Some of the specific measures that help in positioning this dial include:
• Financial “plan” to “actual” performance;
• The quality and accuracy of longer-term financial plans;
• Technology currency in accordance with stated guidelines;
• Security infringements, and resolution of attempted breaches;
• Data management practices.
DESIGNING THE BSC
The primary “audience” for the BSC is the CIO and his/her management team. It is, in essence, their “report card”, but its usefulness increases if unsatisfactory areas of performance (or negatively trending performance) are vigorously pursued to their root cause so that remedies may be applied. Also, the BSC should be the ultimate gauge of the effectiveness of continuous improvement programs in the IT environment.
But others in the organization are also interested in the results and should have input into the design. The CEO and Executive Management Team should provide input into the composition of measures for the “IT Leadership” dial, and perhaps others, such as “Asset Management”. Similarly, the client constituency could provide valuable input on the “Client Satisfaction” dial, and perhaps others, such as “Project Management” and “Operational Performance”. Finally, Human Resources would have important opinions on the composition of the “People Management” dial. Also, all these constituencies would certainly be interested in seeing the results of the BSC at each reporting interval.
PERFORMANCE AND PRODUCTIVITY MEASURES
The BSC represents the highest level of measurement in the IT environment and the ultimate “report card” in communicating throughout the organization how well IT is supporting overall corporate objectives. Several other performance and productivity measures assist line and staff managers within the IT service delivery organization in establishing target levels and in managing results. Most of these will be aligned with and contribute to the position of one or more BSC dials. There may be a few, however, that are quite unique to a certain IT area and have little or no bearing on the BSC.
Figure 2 illustrates, conceptually, the relationships among IT performance/productivity measures and the BSC. Key to this is a central focal point for the collection of operational performance, project performance and productivity measures to analyse and assimilate into the BSC. This implies consistency in data collection, analysis and reporting for all measures used by the various leaders within the IT organization.
INTEGRATING WITH OTHER MEASURES
The design of an effective BSC program within IT must integrate with other performance and productivity measures used in the management of the department. It also provides a test for the meaningfulness of some measures that may have been used for many years.
CIOs may find the BSC technique invaluable in substantiating IT’s contribution to organizational objectives and advancing the esteem of IT professionals involved in its leadership, project and service delivery.
Graham J. McFarlane, P.Eng., ISP, FCMC, is a Director in the Alberta Practice of Western Management Consultants. Since 1978, he has specialized in providing consulting services in the management of information technology.