So, having moved to a new part of the country, in order to help us find new restaurants, my wife and I have been using Google Maps/Google Local as a basic restaurant search engine when we are looking for specific cuisines as well as well-rated restaurants.
Since acquiring ZAGAT, Google has "normalized" their ratings system in a similar fashion to ZAGAT, it has a point scale that is identical. However, the ZAGAT Survey reviews that appear on Google are compiled by ZAGAT using their known amalgamated methods of combining diner surveys and professional reviewers (as they have always been) whereas the Google restaurant reviews are 100 percent user-contributed.
So while in a search both types of reviews may appear right next to each other, a ZAGAT "27" and a Google "27" are very different.
The problem with the Google ratings system is that the ratings are assigned based on a mob-mentality type of approach.
There have been many criticisms of ZAGAT's system and the way it is weighted and what kinds of flaws it has -- historically, we've had some very big threads on this site and my co-founder, Steven Shaw, has written extensively about this.
But I have always felt that ZAGAT had at least enough sensibility and representative sampling imposed on it to make it a good initial point of reference and has a good level of base information to inform the diner about overall restaurant quality, whereas Google's system seems to make no sense whatsover and can empower a single reviewer, in some cases, to artifically inflate or knock down the rating of a restaurant, particularly if there aren't enough ratings submitted on a restaurant to give it a good representative sampling.
I will give you one particular example. This week, my wife and I went to visit two Thai restaurants in South Florida, both located within one mile of each other. The first is Pad Thai in Tamarac which has a 24 rating, and the second is Tobu Thai & Sushi in Coral Springs which until last night had a 27. Today it is a 25. I'll explain why in a moment.
The Google ratings system is composed of three data points, Food, Service, and Decor. Each of these data points is rated from 0 to 3. The overall restaurant rating is computed from an algorithm that essentially takes the total amount of user ratings in each of these areas and averages them.
So if you have a restaurant that only has a few reviews, but they are all exceptional (a 3 in each datapoint) you can easily crank the overall rating up to 27 or higher. And in ZAGAT parlance, a 27 is an exceptional, destination worthy restaurant at very high culinary standards. So you can see how easy it would be to confuse a ZAGAT 27 with a Google 27.
Comparably, a local Sushi or Thai restaurant would never end up being a ZAGAT 27, it would be considerably lower unless they were exceptional examples of the genre. For example, Jasmine Thai & Sushi in nearby Coconut Creek, is a ZAGAT 22. Which is a solid review for a local Thai/Sushi joint under the ZAGAT system.
Conversely, a few bad reviews (zeroes across the board) among only a few total reviews will also deflate a restaurant's rating.
I saw this happen in the case of Pad Thai, which I thought was a very solid, local and authentic Thai restaurant. Probably would also be a 22 or 23 on ZAGAT. One user had given the restaurant zeroes across the board, and I've captured this review for posterity just to illustrate how ludicrous it is that a single user have this much influence.
I flagged this and informed Google, but to offset this we gave this restaurant a deservedly good review. So far it has not moved the needle on the overall rating. I'm hoping they can remove this, because this is a totally unfair way to evaluate a place, especially if you haven't even eaten there.
The other restaurant we went to, Tobu Thai, was a total disappointment in every regard. You can read Rachel's review on the page I linked above. As a result of that review, it was knocked down from a 27 to a 25 overnight.
First, I think it is confusing for Google and ZAGAT to have identical numerical ratings, and they should consider giving Google a different way to identify those reviews. Second, it's obvious that mob mentality in either direction or a low representative sample is not helpful in determining the qualitative aspects of a restaurant.
My opinion is that Google needs "Super Reviewers" or people who have the experience to make informed judgements to supplant the regular reviews, sort of the way Amazon has trusted product reviewers who have a history of reviewing books and items for sale, so prospective diners can make better decisions regardless of what numerical rating the algorithm has given, or so the algorithm can make appropriate judgements/weightings.
Not only should a single reviewer's feedback be flagged as helpful or informative. etc (and I think the reviewer should get negative imput as well if there's a clear misjudgement or lack of expertise, something Google does not do now -- you can only rate a review as "Helpful" or "Not Helpful") but it should also go on their record to allow Google to build a history of confidence, or conversely, indications of bad advice for that particular reviewer. Good reviewers should get gold stars, or plusses, or whatever, just like resellers do on eBay. Bad or seriously uninformed reviewers should get bad marks.
I'd appreciate anyone's thoughts on this.
Edited by Jason Perlow, 31 March 2013 - 11:23 AM.