“Developers develop. That’s what they did.”
That’s how Todd Vernon, CEO of VictorOps, perceives DevOps culture over his career going back to 1997. “They chose that career because they like to do that.”
Through to 2005, he said in a recent online panel, developers spent a great deal of time writing product requirements and even longer writing software with new code pushed to production every three to six months. Once the code worked, “we locked the data centre door and didn’t let anyone in there. That’s largely how reliability was obtained in those days – by not changing things at all.”
Vernon said this wasn’t a great model because it didn’t make companies very responsive, which was actually a competitive advantage. “But it did keep things stable.”
Things have changed, he said, in that development is now the new network operations center (NOC): agile has replaced waterfall, virtual has replaced physical and continuous delivery has replaced a lack of delivery. Ultimately, DevOps is replacing operations, said Vernon. “Not for all companies and not right away,” he said, “but I honestly believe developers are replacing the NOC.”
He said one of the reasons for the shift is that many customers don’t want that first responder to be someone who can’t actually do anything about the problem. Vernon said his company has seen good results from putting developers on the front lines and on call, but it is a significant culture shift.
Kurt Bittner, principal analyst with Forrester Research, wrote a brief earlier this year on the topic of putting developers on the front lines. The firm’s research has found that developers and operations live in different worlds with different cultures and values. “It’s really the starting point for a lot of organizations,” he said. “They have set up a situation in which they have actually structurally separated Dev and Ops.”
In addition, said Bittner, they have created a situation where Dev and Ops are set up do to different things. Development is all about encouraging change through innovation while operations is about maintaining stability by preventing change, or at least, minimizing it. However, a better approach to maintaining stability is to do things that the organization isn’t great at doing more often so it’s better at doing those things.
“We need to change the incentives for both Dev and Ops and focus on customer success,” said Bittner. “If you don’t have innovation your customers go somewhere else.” On the flip side, he said, if you’re not providing stability, customers will also go somewhere else too, even if you are being innovative.
Bittner said the key to merging DevOps into a single culture is measuring everyone the same way. “Part of that means measuring devs on stability and part of that is measuring ops on innovation.” Most importantly, he said, is to reward everyone for improving the customer experience.
One thing that helps developers adapt to the new culture is having to bear the brunt of a new application being deployed before it was fully baked – it allows them to walk a mile in ops’ shoes. “The development equivalent of that is being on call,” said Bittner. “If you’re on call and you’re the one who has to answer and fix it when something goes down, then you’re much more aware of what’s going on.”
There are other benefits to putting developers on call. Rather than having one person being the expert on an application, there is a “tribal” effect of spreading the knowledge, including knowledge of the code and where things are likely to break. Bittner said developers on call will incorporate those experiences when writing code and write better tests. “Over time the code gets better. It gets more resilient.”