When I saw the article this morning in the paper, I had an initial reaction based on the fact that the article did little to help the user understand the issue. It merely reported that Google was the worst. I have since reviewed the study itself, and now understand why the article was so poor. The report itself was hardly an objective view of the complex issues surrounding internet privacy. In fact, I stated in my initial reaction that the problem with the media is that it does a generally poor job of helping people understand the issue and clarify it. So, you can imagine my chagrin when I read the following in the report itself.
The report was compiled using data derived from public sources (newspaper articles, blog entries, submissions to government inquiries, privacy policies etc), information provided by present and former company staff, technical analysis and interviews with company representatives.
The first source of information is ‘newspapers.’ Those of us in the industry are well aware of the inadequacies of the general media when reporting on internet issues. This is hardly a reliable source.
Then there is the ‘governmental inquires’. Sighting:
Generally poor track record of responding to customer complaints. Ambivalent attitude to privacy challenges (for example, complaints to EU privacy regulators over Gmail).
Nothing in the summary or the “detail” of the report addresses relative size or normalization of data, or anything objective at all. It all appears too anecdotal, with no real data point such as % of users, or number of complaints. I am sorry, but having been involved in market research for nearly 20 years, I’d be hard pressed to assess any performance level on “Generally” anything.
Okay, the studies “Assessment” process:
Where possible we present data on specific privacy practices. It was not always possible to precisely assess a company’s approach in each category. As a result, we erred on the side of caution and gave the company the benefit of the doubt and assessed it only for what we could actually identify.
From a methodological perspective, this is absurd. Given that the source of the “data” for this study comes from public sources, this methodology overlooks two big issues. 1) By their very nature, larger or fast growing companies generate a disproportionate amount of ‘public’ information and 2) there tends to be a ‘pile on’ mentality regarding general media as well as blogs and other public sources of information. If you combine the methodology with the data source, you can reasonably expect stagnant companies to rate very highly on this scale.
When I got down to the “Why Google got the worst rating” section, I read that corporate “Ethos” was a factor. Ethos? This is hardly an objective measurement. Google stood up to the government to protect user data (and won) while others caved. You may knock Google on many accounts, but on Ethos? Google is pervasive, and I guard against opening up too much to them. However, I am glad to see that what I do share is defended by Google when they have it.
The privacy issue is real. Unfortunately Privacy International’s approach leaves too many gaps. It lacks objective data, uses openly subjective metrics and lacks a sound methodology. If this is the approach our industry takes on privacy, then the issue itself will not receive the proper scrutiny, the users will remain woefully ignorant and we will find ourselves victims of real privacy abuse.
While I fault PI for publishing this, I place equal fault on the media like Associate Press where I first picked up the story in my local paper. If there had been a proper review of the study, it either would not have made it into the press, or the article would have provided an object view. Our press used to challenge information. Now, unfortunately, it just passes it through.
Matt Cutt’s perspective
Danny Sullivan’s dissection.
