Show us your ethics, or we’ll show you the door. Earn back our trust in your objectivity. Prove the value of your research to our businesses today.
Those were the core messages that some of you delivered to IT market research firms in Computerworld’s user survey on the ethics and disclosure policies of IT research firms. A substantial 87 percent of 133 senior IT managers we surveyed said yes, they have questioned the statistical value or integrity of market research. The same percentage said yes, they want “published, clearly stated ethics policies governing the vendor/client relationships” from analyst firms such as Aberdeen Group Inc., AMR Research Inc., Forrester Research Inc., Gartner Inc., IDC and Meta Group Inc.
IT executives are issuing a clear message. They want greater disclosure about the financial relationships between vendors and market researchers. “Like any other companies, I understand that (IT research firms) have to go after different sources of funding,” said Cathy Brune, CTO at Allstate Insurance Co. “But they need to be honest about saying who pays for the research.”
To their credit, both Forrester and Aberdeen responded to criticisms about “praise-for-pay” business practices with immediate changes. Forrester is no longer letting vendors that sponsor research then publicize the results, as recently happened with Microsoft Corp. and PeopleSoft Inc. Aberdeen will start posting an ethics policy online and disclosing sources of research funding.
The other firms we spoke to for our report — including Computerworld’s sister company IDC — all clarified the rules of engagement on how their research is funded. They also took pains to address vendor-sponsored research, which they recognize as the greatest potential conflict of interests and threat to their integrity. Once lost, user trust in the validity of their research would be forever gone.
Some of the IT executives we interviewed also raised the specter of analysts wielding undue influence over how a fledgling technology market develops. IT researchers “can tout a particular product and have their subscribers go in that direction at the cost of some very good products and services that don’t get the support they deserve,” said Bruce Fadem, CIO at pharmaceutical company Wyeth.
Bottom line: Everyone in this industry should continue to question the source and veracity of information about technology. Hopefully, this heightened awareness of how IT research is conducted will help.
One final note on this topic: Last month, when I first wrote about analyst credibility problems, I cited Nucleus Research Inc., as an unusual example of an independent research firm that relied “entirely on its user client subscription and accepts no vendor business.” That was inaccurate (my mistake), and I overstated the case.
“We don’t do any custom or commissioned work for vendors,” said CEO Ian Campbell, “but to say we never take money from vendors is not quite right.” Nucleus sells its subscription service only to users, but vendors and other interested parties can buy reprints of the company’s ROI reports and case studies. So a percentage of its revenue may come from vendor sources.
Campbell added that vendors also refer their customers to Nucleus to conduct ROI studies for them. In some cases, the vendors have paid for that research on behalf of those customers, although a nondisclosure agreement is always signed with the user. “There’s no question about whom we’re working for,” he added.
Seems like a good thing for everyone to keep in mind.
