This summer, Southwest Airlines experienced a chain of failures that led to cancelled 2,300 flights over four days, costing the airline an estimated US$54 million. Not long after, a power loss at Delta Air Lines cascaded into a massive outage, and British Airways has experience several instances where its check-in systems failed.
Mike Rosen, research vice president for executive programs at IDC, doesn’t want to pick on the airlines, but you don’t want to be in the headlines, he said in a recent webinar, Technical Debt: A Framework for Unfunded, Critical Technical Liabilities. And these “avoidable disasters” are in part due to aging systems that have become brittle due to budget pressures and deferred investments, he said.
“Believe it or not, it could be worse,” said Rosen. These “operational meltdowns” are the extreme culmination of what IDC calls “technical debt,” something IT departments struggle to identify, let alone explain to a company board of directors in a 30-minute presentation so they can understand its liabilities and consequences.
Consider an organization such as Citibank, with 20 data centres, 50,000 servers, 350,000 laptops and desktops, 20,000 applications and 12 billion transactions a month, said Rosen, or Amazon Web Services with its presence in 11 regions, 28 availability zones and 2.1 million servers. “How do we get our heads around all of this complexity, much less communicate?” Even a diagram of applications within an SMB can be daunting, he said.
Non-technical people don’t understand the technology or its complexity, said Rosen, and you can make the argument they shouldn’t have to. But they do need to understand the consequences of technical debt and IT must learn how to communicate it in business terms they understand. The challenge is that technical debt is often not immediately visible.
One of the primary causes of technical debt is the advent of bi-modal IT. As IT extends itself to support new business models as part of an organization’s digital transformation process, IT is often split between fast and slow, the new and the traditional. “A chasm develops,” said Rosen. The fast IT is innovative and responsive, but long-term issues that haven’t been thought through in the race to innovate and are thrown over the wall for operational IT staff to fix. “Technical debt is like financial debt,” he said. “It must eventually be paid.”
Other causes of technical debt include the perceived need to get to market quickly, whether it’s necessary or not, said Rosen. This need for speed cuts corners and created liabilities. Technical debt is ultimately the residual cost of completing technology tasks previously left undone in the race to be agile, innovative, or from lack of funding, he said.
“Most technical debt is not immediately visible,” said Rosen. “We can’t wait for the debt to become visible before we act.” It also compounds itself if not checked, he said, as every change or enhancement made without fixing the debt adds workarounds, inefficiencies, complexities, fragility, and costs. It can lead to consequences as extreme as the outages experienced by major airlines. Technical debt also increases indirect costs and eliminate the ROI of technologies. “Indirect costs are the most insidious.”
For example, it can have an impact on the ability to attract and retain talent. “No one wants to work on crappy systems,” said Rosen.
IDC’s framework for managing technical debt includes several best practices, beginning with making it visible by tracking and recording it in a repository. From there, a quarterly report should be produced with budgeted priorities to resolve that debt.