Quite the storm brewing in the online measurement space. A quick recap:
On Jan. 22 the WSJ’s Peter Kafka blogged about comScore’s plan to charge $10,000 a year for its new hybrid measurement service. When comScore first announced the Media Metrix 360 service last May, many observers applauded its efforts when to upgrade what was viewed as a flawed panel-based method that in some cases dramatically under-reported website traffic.
But the new details about the pricing plan caught the attention of some critics, especially given the growing popularity of free measurement services from companies like Quantcast and Compete. Mahalo CEO Jason Calacanis, extending a long-running comScore rant, called out comScore as “the industry’s biggest bully” and urged publishers to boycott the company’s services.
That led to a lot of tit-for-tat between Calacanis and a host of other players you don’t really need to care about unless you enjoy these types of public peeing contests.
There were two responses of note: comScore CMO Linda Abraham defended the pricing decision on the company’s blog on Jan. 24:
“While it’s clear that there is widespread consensus that building Media Metrix 360 is the right approach, a small minority have questioned our decision to charge non-clients $5,000 to set up their site’s reporting using the Media Metrix 360 methodology. We believe this is a very reasonable price to charge for six months of reporting back to the client. If at the end of that period, the client does not want to continue subscribing (no-one has yet done so), we will still continue to implement the 360 methodology free of charge as long as the web site continues to maintain our beacons / tags.”
She also took a shot at competitors that don’t charge for their measurement services:
"Other services monetize their approach … by selling publishers’ audiences (using their own cookies) to advertisers (and occasionally to their competitors), sometimes with the owner of the cookies not even being aware that this ‘Trojan Horse’ strategy is happening. comScore is not in the business of selling a client’s cookies and that is part of the reason why our industry views us as the leading unbiased yardstick of Internet behavior. In life and business, there is no such thing as a free lunch, so Web site operators need to carefully consider the true cost of being beaconed by companies who are in the business of selling their cookies or using them for targeted advertising by leveraging information they got from those sites.”
This, of course, led Quantcast to respond, even though Abraham did not mention them by name. From Quantcast’s corporate blog:
“We do make money from advertising services, by helping major media companies create more compelling audience packages they can sell to advertisers. … The Internet is highly fragmented and makes extensive use of syndication and 3rd party distribution – this is one reason panel based solutions don’t work (there are more). In the three years since Quantcast launched the first direct measurement audience service, Comscore have [sic] labeled our service as harmful and our approach as rogue. With their current attempt to replicate it, it would appear we might be onto something after all. The free market favors innovators.”
The main issue here is that comScore will end up with bifurcated reporting that will effectively penalize websites that don’t subscribe to the new hybrid measurement service. For now, comScore is providing parallel data sets to clients: hybrid and panel-only. But beginning May 1, according to Josh Chasin, comScore’s chief research officer, comScore will provide only the hybrid syndicated data set. This means that comScore’s website rankings will mix two sets of data: hybrid data for sites that are paying for the hybrid service, and panel-only data for sites that are not.
Apples to oranges
These apples-to-orange comparisons could cause serious issues for sites that don’t pay for the hybrid approach, since their traffic numbers may be much lower than competitor sites that are using the hybrid method.
“Their old approach at least was consistently off” for all websites, said Chris Wilkes, vice president of marketing and audience development for Hearst Magazines Digital Media. “The hybrid, if it’s adopted by some but not all, makes those comparisons that much harder to do.”
While comScore certainly has a right to charge for its services, it's easy for publishers to infer a more sinister message: Pay for our new, more accurate service, and your traffic numbers will improve. Don't, and your numbers won't stack up favorably against our clients.
The Internet Advertising Bureau, which for years has been advocating more transparency and better methods for measuring Web traffic, applauded the progress comScore is making with its methodologies but criticized the inability of the parties involved to develop common measurement standards.
"The IAB believes that hybrid measurements will naturally get marketers, agencies, and media much closer to measuring real marketing effectiveness than any simplistic reliance on only panels or only servers,” Sherrill Mane, the IAB's senior vice president of industry services, said in a statement. “At the same time, the gyrations and controversies over these belated attempts to introduce ‘improved’ measurement systems reinforce that industry's systems and processes for measuring such vital ‘currencies’ as audience size and composition have been and remain broken. The time is long overdue for marketers, agencies, and media to work together to simplify and standardize interactive measurement."
Well put. The measurement methods from comScore and others will always be a moving target until and unless the industry agrees on standards that everyone can conform to. If publishers continue to allow themselves to be judged solely by this fuzzy math, they deserve whatever abuse they get from service providers and media buyers. Ultimately, publishers will be better served by moving beyond volume metrics and working with their advertising partners to develop better ways to quantify the value, not the size, of their audience.
