IT pros do battle every day — with cyber attackers, stubborn hardware, buggy software, clueless users, and the endless demands of other departments within their organization. But few can compare to the conflicts raging within IT itself.
Programmers wage war with infrastructure geeks. IT staff butts heads with IT management. System admins battle for dominance. And everybody wishes security would just leave them alone.
“One of the iconic examples of IT conflict is operations versus software development,” says Mark White, CTO of Deloitte Consulting‘s Technology practice. “The software dev people build something and throw it over the wall into production, and operations is expected to make it work 24/7. Or the apps developers versus the database managers. App dev wants a high level of abstraction, but the database admins need well-conceived calling patterns to get the performance they need. And the security people, well, it’s sometimes said they put the ‘no’ in ‘innovation’.”
Are you at war with other members of your IT staff? Read on to see if these stories sound familiar and to find out how to transform tension into productivity for all.
IT turf war No. 1: Security vs. the rest of IT
When Jon Heimerl worked at the CIA, his job was to ensure that data passed securely between the overseas sources who gathered it and the stateside analysts who parsed it. Trained as a systems engineer, Heimerl regularly worked with one of Langley’s many security departments. On the wall, next to the door leading to this security department, was a sign that read, “The answer is no.”
“When someone comes to the security people and says, ‘I want to do this,’ security’s default answer is to say no,” says Heimerl, who’s now director of strategic security for managed security services firm Solutionary. “It’s not that they want to keep people from doing anything. They just want you to think about it first so that you can do it more securely. But the perception becomes that security just says no.”
It’s a classic IT conflict. The security wonks believe users can’t be trusted. The techs can’t get any projects off the ground because security has tied their hands. Both sides circle each other warily. Worst-case scenario: Employees decide to bypass the House of No entirely and do things on their own.
Heimerl says Solutionary was once called in to work with a Fortune 100 retailer whose marketing department had decided, on its own, to build a public-facing website without informing IT security. The site was outside the company’s firewall and employed default user names and passwords. It was only discovered after Solutionary did a routine sweep of the company network.
“From the time we started pen testing it until the time we had administrative privileges was about 17 minutes,” he says. “From there, we had a free pipeline into the corporate network. Within about 30 minutes we were able to log on and change prices for items on the websites of individual stores.”
When they heard about it, Heimerl says, the retailer’s IT security department went ballistic. But rather than simply spank the marketers and shut down the system, Solutionary persuaded them to build a new site that did what marketing needed, in a safe and secure fashion. Though it caused some angst, Heimerl says in the end everyone got what they wanted. Marketing had its site, and security was able to safely support marketing’s business needs.
The good news: Wider adoption of cloud computing has actually eased the tensions between the IT security group and the rest of IT, says Tomas Soderstrom, CTO for the Jet Propulsion Lab‘s Office of the CIO. Other groups can’t afford to lock out IT security, so instead they bring them in from the beginning, he says.
“Under the leadership of our CISO, Jay Brar, IT security at JPL has become very proactive,” he says. “We used to say that the buck stopped with security — they could shut you off if they had to. We’ve changed that to ‘the buck starts here.’ The job of IT security is to protect by enabling. Because if they don’t, the business side will do it any way they can, and you end up with systems that are not secure.”
IT turf war No. 2: Ops vs. dev
One side of your IT department is laser-focused on keeping your systems up and your costs down. The other side wants to push the envelope until it bursts. Welcome to the war between your ops squad and your dev team.
“>IT turf war No. 2: Ops vs. dev
One side of your IT department is laser-focused on keeping your systems up and your costs down. The other side wants to push the envelope until it bursts. Welcome to the war between your ops squad and your dev team.
“>IT turf war No. 2: Ops vs. dev
One side of your IT department is laser-focused on keeping your systems up and your costs down. The other side wants to push the envelope until it bursts. Welcome to the war between your ops squad and your dev team.
“The classic conflict is that IT is very often just managed as a cost center,” says Ted Shelton, CEO of Open-First, a consultancy that helps Fortune 500 companies manage disruptive technologies. “They believe their job is to figure out how to do more with less. And when management is looking for places to cut costs, IT is one of the first to get squeezed.”
Shelton says he recently met with a large retailer whose development side was building out the retailer’s Web infrastructure and wanted to capture the clickstream from the Web for in-depth analysis. The retailer’s operations side said sure, you can capture all those clicks, but at the end of the day we’re going to throw them all away, because we won’t want to pay for the cost of storing that data. The developers were forced to go with a third-party vendor.
“The operations side saw its job as managing costs,” says Shelton, “while the developers saw their job as managing quality. They said, ‘We can’t deal with these ops guys anymore, let’s go around them.’ It is amazing to me that in 2011 people are still worried about the cost of disk space. But this a common mistake across industries where IT is managed as a cost center and not something that can give companies a real competitive advantage.”
But it’s not all the fault of ops. The development side also shoulders its portion of blame.
“Let’s start with the notion that app developers do not set out to build network-friendly applications,” says Steve Shalita, vice president of marketing for NetScout Systems, which provides unified service delivery management services. “These apps aren’t optimized to enable the network to run efficiently; they’re built to do what they’re supposed to do. So the dev guys create the application, throw it over the wall for the application implementation team to deploy it. The network guys just provide a connection. Neither side is working together to tune the app or optimize the environment.”
That’s when things break down and the finger-pointing begins, Shalita says.
“Five years ago the network was probably to blame 60 percent of the time,” he says. “Now it’s largely the apps that are the root of the problem. But everyone still says, ‘Darn, this network is slow.'”
At that point, it becomes an endless loop, says Steve Mills, senior networking engineer for IT services and consulting firm Jelecos.
It starts when something breaks, he says. Engineering is called in to fix it, and they find third-party apps installed on their servers. They immediately take away the developers’ access rights. Management notices a drop in productivity (or gets tired of hearing complaints from the dev team) and sets up a meeting with engineering. The engineers are instructed to restore the developers’ access rights. Then something breaks.
Repeat onto infinity.
“The problem is that in most cases IT departments still operate in silos,” says Shalita. “There was a time when developers were asked to build an app and the company would create the infrastructure to support it, no matter what the cost. Organizations don’t do that any more. So both sides have to operate more intelligently. Developers need to engineer applications to operate more efficiently in their environments, and they need real-time data from operations to tune their apps to work better.”
IT turf war No. 3: Admin-on-admin violence
For every few thousand hardworking, conscientious system administrators, there’s one who will abuse his awesome powers. That’s why one of the biggest battles within IT departments is between the good admins and the evil ones.
The classic scenario: A sys admin departs on bad terms and decides to wreak revenge.
Solutionary’s Heimerl says he worked with one high-tech company that terminated an admin for selling pirated satellite equipment via one of the company’s Web servers. As he cleaned out his desk, he opened one file — the one containing the corporate escrow key for all the encrypted files held by the company, as well as all of the employees’ encryption keys — deleted its contents, and resaved the empty file, making it unrecoverable. Roughly two dozen employees lost complete access to all of their work for the previous three years, he says.
Fortunately, Heimerl says, such admins are the exception and not the rule.
“In 25 years of doing this job, I’ve dealt with hundreds of companies employing millions of people,” he says. “We’ve had to deal with admins who’ve abused their privileges maybe a couple of dozen times. For 99.999 percent of admins, this is never the case. But when we do hear about a rogue administrator gone wild, the danger is to say admins are going to run amok and steal things from the company. It breeds a culture of mistrust from management to security to IT. It’s counterproductive. But you can’t afford to be unprepared for it either, or it could cause an event that can cripple your business.”
Independent technology consultant Allan Pratt says he started his tech career as a sys admin for a small newspaper back in the 1980s. The guy he replaced had taken all the system passwords with him when he left, leaving the company to start from scratch.
“I think some people you’d call bad sys admins may just be people who don’t think,” he says. “They aren’t self-aware. They leave and they don’t care about anything else. But it only hurts them in the end. Down the road someone will ask, ‘Do you know so and so?’ and you’ll say, ‘Yes, he walked out on us and took our passwords with him.’ It’s a small industry. The only things that have meaning in this life are your name and reputation. Lose them and you’ll never get hired again.”
Open-First’s Shelton says bad employees are inevitable, but companies can do more to wrest some of the power from a single person’s hands.
“It’s the human condition,” he says. “People who have a little bit of power and feel better about themselves when they exercise that power. It’s as true of a guy running a parking garage as it is of a systems admin. But it’s also an opportunity for organizations to look at the points of control and devolve responsibility so that multiple people are involved in all the key decisions.”
IT turf war No. 4: IT management vs. IT staff
Most geeks wouldn’t recognize a critical business process if it bit them on the nose. And though their boss may have”technology” or “information” in his job title, he appears to knows little about either. This is perhaps the most intractable battle in all of IT — the war between the officer corps and the troops.
“The biggest conflict is between IT management and IT staff,” says Pratt. “For some reason, the companies I’ve worked for seem to hire or promote people who are not technologically literate. It’s like that person lost a bet or the president of the company has a half-wit brother who needs a job. You have the IT guys in the field saying, ‘You really need to do XYZ,’ and the managers saying, ‘We’re not going to do that; it’s going to cost too much money.’ They’re constantly blocking things that have to be done just because they can.”
Back in the ’90s, Pratt says he worked for a monitor manufacturer based in China, whose CTO decided to ship all of its CRTs to the United States without doing any burn-in or testing. “Our IT guys were saying, ‘No, you got to do testing first.’ They wouldn’t budge. Sure enough, the monitors showed up in California with skewed pictures. A good 80 percent of them were returned.”
On the other hand, says Shalita, the most important decisions a CIO faces aren’t about technology per se, but about business outcomes. And that may never enter the mind of an in-the-trenches IT grunt.
“I’ve had a lot of discussions with a lot of very tech-savvy CIOs,” he says. “But at the end of the day, the business decisions they need to make aren’t based on sexy technology — they’re based on business outcomes. There’s pressure on the CIO from the CEO to deliver business value. The IT guys are focused on the technology in their particular tower.”
The fault usually lies on both sides of the divide, says Peter Marsack, vice president at Vision Computer Solutions, an IT service and support provider.
“Most of time time I see conflict relating to IT staff conforming to company policy or believing in the concept of company hierarchy,” he says. “The top sources of conflict are the tech person’s ego, poor management, a lack of proper leadership, and allowing technical people to make business decisions. The solution there is to know your role and let your talents shine where they should.”
And keep your head low, to avoid getting hit by friendly fire.
