Zagat grade inflation

[Fat Guy]

The March issue of SmartMoney magazine contains a feature on the Zagats, and makes some interesting contributions to the discussion of the Zagat guides. Being a money-oriented magazine, SmartMoney did actual statistical analysis of the Zagat ratings over time and came to the following conclusion:

Over the course of several months, SmartMoney tracked down former Zagat editors, crunched the guides' numbers and discovered a few surprises about how the Zagat sausage is made. Among the most important: a curious case of grade inflation.

. . . . .

When the Zagats started selling their 1983 New York restaurant guide, it was no mean feat for a chef to score a food rating of 20 or higher, the benchmark for "very good to excellent" in Zagat terms. Only one in four New York restaurants did so at the time. Today fully 70% reach those heights. It's as if the bottom tier dropped out: Just over a decade ago 189 out of 1,300 New York restaurants rated 15 or below; today only 23 do, despite the fact that the guide now rates more than 1,500 restaurants.

The Zagats respond in a couple of ways. They say they emphasize recommended restaurants, and that restaurants have gotten better. But it's absurd to think those factors justify a shift from 25% of the rated restaurants scoring 20+ to 70% reaching that level.

Definitely read to the end of the piece to get to the anecdote about Tim Zagat storming out of an interview. (There's also a quote from me somewhere in there).

[JohnL]

I think the key is the advice offered at the very end of the piece. "It's still a real handy book, but I think people should take it with a grain of salt..."

I suspect that the people who participate in the reviews and ratings face the dilemma inherent in any rating system, can a pizza joint be rated along side a high end establishment? I have never had a problem with this, it is obvious to me that a score relates to a restaurant within its peer group.

I also wonder if the general public is all that discerning. People who frequent the Cheesecake Factory often probably believe it is a twenty point or more establishment, it is popular and it probably does do a better job than most of its competitors but....

So, I think that using Zagat's (or any guide) one must bring some common sense, some outside knowledge and try to put the guide into as much context as possible. Anyone who takes the numbers literally, is IMOP, being naive or foolish.

Now let's see--does three stars from Bruni equal a twenty or a twenty five in Zagat?

[Endy']

I don't understand why ratings get the discussion that they do. When I read local restaurant reviews in Toronto Life, I go straight for the text. At best I'm peripherally aware of how many stars were awarded.

but when the bulk (or entirety) of a review is a star or point rating, it becomes completely meaningless to me. Unless I have the exact same tastes as the reviewer, I'm unlikely to agree with their star rating. So I need to know WHAT about a restaurant it is that the reviewer liked, so I can decide whether I might like it.

how many of us come to eG to post about a meal we've had? And how often are those posts "7.528/10" or "****"? I don't think humans do (or can) think of restaurant experiences in terms of a number.

incidentally, I did try reading Zagat reviews a few times but found they were too focused on their aggregate rating, usually only justified by snippets of quotes from multiple reviewers. I don't have "average" tastes so none of this was helpful to me.

[Deacon]

The numbers are the easiest thing about Zagat. You don't get any REALLY useful information from those little one paragraph summaries because the "quotation" marks make everything very "irritating" to "read."

To address Fat Guy's original point: grade inflation happens everywhere. You can see it in Zagat ratings, in the grades teachers give students, in bonuses given for exemplary performance at work. The tendency creeps over time in the direction of giving an award to everyone. And of course, when everyone gets a high grade, the grade itself becomes meaningless.

Edited February 28, 2007 by Deacon (log)

[rconnelly]

The numbers are the easiest thing about Zagat. You don't get any REALLY useful information from those little one paragraph summaries because the "quotation" marks make everything very "irritating" to "read."

To address Fat Guy's original point: grade inflation happens everywhere. You can see it in Zagat ratings, in the grades teachers give students, in bonuses given for exemplary performance at work. The tendency creeps over time in the direction of giving an award to everyone. And of course, when everyone gets a high grade, the grade itself becomes meaningless.

←

Exactly! We wouldn't want anyone's feelings to get hurt, would we?

[Busboy]

Actually, the real grade inflation is from city to city. I won't pick on any town by name (OK, I will), but if there were a Zagat for Carson City, Nevada, you'd find restaurants getting the same scores and the same accolades as Per Se or Alain Ducasse. Say what you will about the Micheline Guide it brings a decent measure of consistency across borders.

[Deacon]

Actually, the real grade inflation is from city to city.  I won't pick on any town by name (OK, I will), but if there were a Zagat for Carson City, Nevada, you'd find restaurants getting the same scores and the same accolades as Per Se or Alain Ducasse. Say what you will about the Micheline Guide it brings a decent measure of consistency across borders.

←

In a way, though, that makes sense. You choose a place to eat from the available choices where you happen to be at the time, so those restaurants are measured against each other. You don't choose from restaurants all over the country when you're deciding where to eat on a particular evening.

[rooftop1000]

the best thing about it is that its easy to carry for the addresses

going to NYC tomorrow...where is my copy?.....

tracey

[Pan]

You can always use superpages.com and menupages.com for the addresses, if you will have access to a computer.

[hjshorter]

Actually, the real grade inflation is from city to city.  I won't pick on any town by name (OK, I will), but if there were a Zagat for Carson City, Nevada, you'd find restaurants getting the same scores and the same accolades as Per Se or Alain Ducasse. Say what you will about the Micheline Guide it brings a decent measure of consistency across borders.

←

In a way, though, that makes sense. You choose a place to eat from the available choices where you happen to be at the time, so those restaurants are measured against each other. You don't choose from restaurants all over the country when you're deciding where to eat on a particular evening.

←

Yes, but it's ridiculous to think that a place in Carson City is going to be then equivalent of Per Se or Alain Ducasse. I have learned to give the Zagat rating about as much weight as I give the local paper's food critic, wherever we happen to be.

[hjshorter]

Actually, the real grade inflation is from city to city.  I won't pick on any town by name (OK, I will), but if there were a Zagat for Carson City, Nevada, you'd find restaurants getting the same scores and the same accolades as Per Se or Alain Ducasse. Say what you will about the Micheline Guide it brings a decent measure of consistency across borders.

←

In a way, though, that makes sense. You choose a place to eat from the available choices where you happen to be at the time, so those restaurants are measured against each other. You don't choose from restaurants all over the country when you're deciding where to eat on a particular evening.

←

Yes, but it's ridiculous to think that a place in Carson City is going to be then equivalent of Per Se or Alain Ducasse. I have learned to give the Zagat rating about as much weight as I give the local paper's food critic, wherever we happen to be.

←

Michelin is far more reliable, even for bib gourmand type places.

[tommy]

You can always use superpages.com and menupages.com for the addresses, if you will have access to a computer.

←

there are many ways to get addresses of restaurants, even without a computer. the point is that it's a relatively comprehensive general-purpose list of restaurants with addresses and phone numbers that is easily slipped into a purse or bag.

[Bricktop]

I get far more useful information from sources like here than from books like Zagat's.

Compare it to Parker's Wine Advocate. There you have a number rating sure,

but the text is the key. The "text in Zagats" is "really just a joke".

It has "no value at all" when "compared to even review posts here on eGullet".

It's useful as a compendium of phone numbers and addresses though.

As far as the grade inflation goes, "If everybody's somebody, then no-one's anybody."

Edited March 7, 2007 by Bricktop (log)

[nikkib]

In the uk Michelin is far more widely used than zagat, you only ever really see the us tourists using them. I found michelin really useful in NYC soon although i know it still only covers a cple of us cities.