Abebooks Inc., an online book trader in Victoria, B.C., installed Agitar Software Inc.’s developer testing tool to quell a fight starting between Abebooks’ application engineers and the quality assurance (QA) department.
According to Jayson Minard, Abebooks’ CIO, the developers were sending applications to QA that still had low-level bugs in them. QA would have to test for basic code errors, giving the testers little time to examine the apps for usability and workflow.
“There was a lot of contention…about who was responsible for quality,” Minard said, outlining the argument between QA and the engineers. Each seemed to think quality was the other’s purview; Abebooks would have preferred both sides to consider quality when creating and testing new applications. “The company was seeing a lot of issues with our production Web sites and our back-end systems — missed bugs that could have been caught earlier.”
Abebooks decided that developer testing was the answer. This software-creation set up would see the engineers testing code as they wrote it. That would improve the quality of the applications and allow QA to focus on quality beyond the code itself.
But developer testing means a lot more work for app builders. “The total manual effort to get our legacy code tested, as well as new code, was incredibly high,” Minard said. The firm needed a developer testing product that would examine the code as the engineers wrote. Abebooks also sought something that would also spell as little work as possible added to the developers’ to-do lists.
Abebooks turned to Agitar for a solution. The Mountain View, Calif.-based tech provider offered Agitator, its automated developer testing software. According to Kent Mitchell, Agitar’s director of product management, Agitator creates tests on its own. It analyzes the code and figures out what tests need to be performed to put the code through its paces.
“It understands certain coding patterns, the structure of the code,” Mitchell said of Agitator. Based on that structure and “using some of its own internal rules, it’ll come up with a set of input data that it thinks it needs to do to test the code.”
After examining the code, Agitator essentially shows the developer what happened when it tested the app. Then it’s up to the engineer to decide whether the software she’s working on needs tweaks or not.
Minard said Agitator, which Abebooks employs not only at the developers’ workstations, but also in the automated build, has helped his company improve application quality. It’s top-of-mind now. The engineers know “there’s this massive monster out there every night waiting to jump on their code, and tell them what the code really does.” That makes it hard to ignore how important quality is in code generation.
Meanwhile, QA has more data to decide what sort of tests they’re going to perform, beyond the developers’ own examinations. “Are they going to focus on edge cases? Primary cases?” Minard said. “They need to pick their spots to have more impact. They’re much more efficient at their work as a result of that.”
The upshot: relations between QA and the developers are better than they were before Abebooks started using Agitator, Minard said. The two sides seem to be working together.
But it wasn’t easy bringing Agitator in. “There was a little resistance,” Minard said. Some of the Abebooks employees weren’t convinced that Agitator would help, or that it wouldn’t mean more work on their part. “Every time you do a big change like this you’re going to have people questioning it.”
Minard and other tech leaders at the company stuck to their mandate for high-quality code, and supported that focus with patience. “We pretty much agreed we’d be relentless about…getting quality on everybody’s mind, and getting these toolsets in.” If development slowed because a developer had to learn how to use Agitator, that was OK.
Abebooks figured it was best to give users time to understand the software. “We couldn’t ask everyone to learn this new toolset and then not back them up on the management side by saying they had to do it in unrealistic conditions.” Agitar’s Mitchell said.
Agitator costs approximately US$4,000 per developer. It may sound expensive, but in his assessment, bugs cost about $2,500 to fix. Agitator pays for itself in two bugs per developer, he said. Minard said Agitator works well for Abebooks. “We really like having something out there watching our backs.”
QuickLink 056899
Related link:
