Microsoft Corp.’s new browser ballot screen, which is supposed to randomly scramble the positions of the top five browsers, instead gives Google’s Chrome the best chance of landing in the preferred first spot, an IBM Corp. software architect said today.
“This was a rookie mistake,” said Rob Meir, who works for IBM and has a degree in astrophysics from Harvard University. “I was definitely surprised to see an error of this type [in the ballot].”
The browser ballot, which began to show today in the Windows Update queues of European users, was mandated by an agreement Microsoft reached last year with European Union antitrust regulators, nearly two years after Norwegian browser maker Opera filed a formal complaint. The ballot appears on Windows PCs where Internet Explorer (IE) is set as the default browser, and lets users download and install rivals, including Chrome, Firefox, Opera, Safari and others.
According to the deal Microsoft struck with antirust officials last December, the ballot screen is to scramble the order of the top five browsers, a change from an earlier Microsoft idea that browser order would be alphabetical by maker. Several rivals blasted that plan because Apple ‘s Safari, which has a very small share of the Windows browser market, would get the favored first position at the far left. “These five web browsers will be displayed in random order each time the Choice Screen is presented,” stated the agreements that Microsoft and the European Commission signed ( download Word document )
Weir said that the randomizing process in the ballot was flawed. “They fell into a well-known trap,” he said in an interview today. “This doesn’t randomly shuffle the positions.”
Microsoft failed to use an established random shuffle algorithm — think of it as the shuffling of a virtual card deck, said Weir — and instead made what he called a “rookie” mistake of sorting an array with a custom-defined comparison function.
“(This) is more in the nature of a ‘naive algorithm,’ like the bubble sort, that inexperienced programmers inevitably will fall upon when solving a given problem,” Weir explained in a long entry to this personal blog on Saturday. “I bet if we gave this same problem to 100 freshmen computer science majors, at least one of them would make the same mistake.”
What surprised Weir was that the mistake wasn’t caught. “For an error like this to happen, all it takes is one person to make a rookie mistake. But for it not to be caught…, that shows it’s a process problem. So you can’t blame this on just one person.”
To determine whether the ballot order was actually random, Weir replicated tests first done by the Slovakian technology site DSL.sk , which was first reported in the U.S. by TechCrunch . Weir took the randomizing JavaScript code from the ballot Microsoft posted on browserchoice.eu , added some test driver code and ran it 10,000 times in Internet Explorer.
Like DSL.sk, Weir found that Chrome was more likely than any other of the top five browsers to show up in the first three spots on the ballot, that IE had the least chance of any to grab one of the first four positions, and that 50 per cent of the time IE appeared in the fifth spot at the far right.
Firefox, meanwhile has a smaller chance of getting the first spot than does Opera, a browser that has about one-tenth the browser usage share of Mozilla’s application.
The fix is relatively simple, said Weir, who spelled out his recommendation in the blog post. By using the Fisher-Yates Shuffle algorithm, Microsoft would actually randomize browser position in the ballot.
Microsoft today declined to answer questions, saying that it had not yet tested or examined Weir’s findings. For the record, a spokeswoman said, “In accordance with our agreement with the EC and the ballot vendors, the ballot screen results are based on a code that randomly generates the results.”
The European Commission did not reply to questions about the randomization of the ballot and whether officials there would investigate Weir’s claims or require Microsoft to revamp the ballot. By the terms of the agreement, Microsoft promised to make changes to the ballot when asked.
“This flaw means that you’re not going to get random results,” concluded Weir. “I am astonished that the bug got as far as it did. This should have been caught far earlier, by Microsoft, before this ballot screen was ever made public.”
Weir has posted the HTML file used for his tests ; users can run the test by entering an iteration count. He cautioned, however, that different browsers would generate different results. To most accurately mimic the actual ballot, his HTML file should be run with IE.