Let's create a better Restaurant Rating System

[jhlurie]

Ron Johnson's question in If the Times is Wrong, Who are the 3 and 4 stars in NYC? thread has got me thinking about how bogus most restaurant rating systems are.

Let's face it... most of the existing benchmarks out there are ludicrous.

Here are some snippets of my thoughts in that thread to spark this debate:

First of all, do you mean "fine dining" establishments, or does your desired version of "the best" recognize that something can be the absolute best of its type but not measurable on the same scale as a fine dining experience?  In this vision, is there such a thing as a four star hamburger, which exists on a different plane, or is everything absolute?

In other words, how can (or should) a rating system discriminate between levels or types of food?

do you envision all aspects of a restaurant to be included within a single rating, or is the gauge you desire only for the absolute quality of the food?

So... should a rating system aggregate a total score, or should each aspect be identified clearly and seperately--and if so HOW?

Another thought... should the ultimate restaurant rating system recognize regional differences, or once again should it be on an absolute scale?

And as a side argument... are there reasons other than tradition that outlets like the NY Times stick with their current systems? Or are there valid arguments for a simpler system? If so, I'd like to hear that too.

Edited March 1, 2003 by jhlurie (log)

[Steve Plotnicki]

It has to be absolute. You are rating the restaurant, not individual dishes. DiFara's might have two or three star pizza but the place is a dump. Most people want to know what type of place they will be dining in.

Let me ask this another way. Isn't the first thing you consider about a meal is whether it meets the social situation you require? Last night I ate at Effida, the new Turkish run by what's his name (I forget.) It is billed as Turkish fast food. But it is really just steam table food. That fact limits the type of occassion I will use that restaurant for in the future. Steam table is a more casual way of dining then if the food was cooked to order. To buck that logic, it would have to be the most phenomenol food in the world. But that isn't very likely when it was prepared at 5:30 and I was there at 9:30. It is also why the dishes happen to cost $4.50 a plate.

[glenn]

Assuming a concensus could be arrived at [yeah, right], how would the system be utilized, or is it just for fun?

[Suzanne F]

Just off the top of my head -- I'd love a system that includes a formula by which

- the quality of the ingredients,

- the availability of food-appropriate beverages,

- the neatness of food presentation,

- the cleanliness of all public areas,

- the attitude, willingness, and circumspection of service,

- the noise level, and

- the lighting level

are all factored in according to a standard set of criteria. Then that number is multiplied by the inverse of the average dinner price. This levels the field, so that an inexpensive, "ethnic" (forgive me) that caringly serves excellent food made from less-fancy (but good) ingredients still has as much of a chance to be rated as highly as a "fancy" place that charges much more.

Each of the these categories can be quantified on an absolute scale -- as Steve P. just said, it has to be absolute. But in addition to the final rating, people have to know the individual components as well. So they can decide whether to put up with dumpiness for excellent pizza.

Anyone want to discuss, or refute? Fine with me.

[Steve Plotnicki]

are all factored in according to a standard set of criteria. Then that number is multiplied by the inverse of the average dinner price. This levels the field, so that an inexpensive, "ethnic" (forgive me) that caringly serves excellent food made from less-fancy (but good) ingredients still has as much of a chance to be rated as highly as a "fancy" place that charges much more.

A level playing field is useless information. Not only to me, but I believe to most diners. Because if it wasn't useless, that is how newspapers and magazines would calibrate their reviews already . The purpose of a restaurant review shouldn't be to drown out individualism, either when describing different cuisines or when talking about people's pocket books. . Let's face it. Daniel cost more then Grand Sichuan and there should be no penalty for price point which is ultimately what you are describing. Any system that takes that into consideration and penalizes you is just implementing reverse snobbery.

[Pan]

Unless good values are rewarded with some kind of check mark, rather than penalizing less good values that are nevertheless fine restaurants.

[Lord Michael Lewis]

One could always try reading the review and making one's own judgement based on the information therein.

[Suzanne F]

What a unique thought!

[mamster]

The Seattle P-I gives separate ratings for food, service, and ambience. Is that a step in the right direction or wrong?

[glenn]

The Seattle P-I gives separate ratings for food, service, and ambience.  Is that a step in the right direction or wrong?

So does Zagat. + cost.

[Basildog]

How about rating a restaurant on what it sets out to do? By this i mean if a Restaurant aims for the finest dining experience possible and suceeds..then rate it 100%

Like wise, a reliable place thats not pushing the boundaries of haute cuisene, but does deliver what it sets out to do..then also rate it 100%

We then just have to define what the restaurants are trying to do

Fast Forward 5 years...egullets rating system( whatever that may be) becomes the worlds leading resturant guide...We have the people, why not?

[Fat Guy]

I'm in favor of rankings as opposed to ratings, and I think rankings work best when like restaurants are ranked against one another and unlike restaurants are ranked in separate categories. A good system would be to take the various styles of cuisine and break each into high-end, middle-market, and cheap-eats groupings. Within those groupings, rankings make quite a lot of sense, and the groupings themselves provide much of the key information most people want to have about a restaurant.

[Pan]

I'd love to see your rankings, Fat Guy. Are you game?

[Steve Plotnicki]

A good system would be to take the various styles of cuisine and break each into high-end, middle-market, and cheap-eats groupings.

You mean like my 2001 Guide to Dining.

Rankings are good for a certain kind of user. Ratings work better for a broader base of users. Most people who have limited time to do research, want to know what the top restaurants are and that's it. I have no way of knowing but, I bet the list of the 25 most popular restaurants at the beginning of each Zagat's is extremely useful for people who want to eat well, but aren't maniacal about every last detail. If you used that list and nothing else, you wouldn't starve. And I would think that describes most of the people who use that type of information. As for Cheap Eats groupings, I really don't think they are that important to people. The great ones manage to make it onto the lists with fine restaurants. For people coming to NYC from out of town, how many Thai restaurants do you need to list, two or three? More then two or three kosher delis? Outside of fine dining, there aren't really that many great places to eat. Two or three in each category is more then sufficient, with the exception of Chinatown because there are so many prolific choices. But even there, isn't five or six choices enough?

Edited March 2, 2003 by Steve Plotnicki (log)

[Explorer]

Random thoughts:

1- One of the reasons I was first attracted to eGullet was that I hoping to find some ranking/rating list somewhere. Fetching the threads can be dreadful sometimes.

2- I favor categorizing restaurants; i.e. listing them inside 3-4 well defined categories. It would be like splitting hair if you try to rank 10 restaurants in a "A" category for e.g. as it would be tough to rank the 20 3-star Michelin restaurants in France. They all are great but also very different at the same time. Descriptive info about them becomes more important than a number.

3- Universality is tough to achieve; for e.g. the way restaurants are reviewed in France is different from North America. That's why Zagat's ratings in France are even more meaningless than the ones in North America and vice-versa Michelin touches only a few restaurants in North America.

4- Finally, what's more important to me is the real-time aspect of the review; i.e. its combined accuracy and recency. Only the Internet can provide that unless you print a guide every month. This is more of an issue for North America as consistency can change often whereas in Europe, there is a bit more of a sense of continuity and slower change.

To illustrate subjectivity and diversity, I just finished watching 2 episodes of the Iron Chef does France with a French Chef vs. a Japanese Chef and they had 2 French judges (Robuchon and Troisgros) and 2 Japanese ones. In each case, the French judges gave more points to the French Chef and less to the Japanese one whereas the Japanese judges did the opposite. Some of the comments were telling: The French judges said they liked the vegetables more cooked when the Japanese Chef made them crunchy [i thought it is a known fact that the French like vegetables well cooked; but how could he make 2 of them more cooked than the others? Then, when the Japanese judge thought the French-cooked lobster was overcooked and therefore tough, Troigros was quick to observe that these Normandy lobsters are naturally tough, hence their texture was well exhibited in the cooking. Go figure!

[Pan]

Does anyone know whether Michelin has any Red Guides for places outside of Europe online? They're not in the viamichelin site, which is just Europe, and Michelin's U.S. site seems to be only about their tire business. I'd find it interesting to see which New York restaurants are listed and how they fared.

[Steve Plotnicki]

Michelin only publishes guides for countries in Europe. Remember they are a tire company and they started as guides for French drivers. It isn't easy to drive your car from Paris to Milwaukee.

Edited March 2, 2003 by Steve Plotnicki (log)

[Explorer]

As far as I know, I have only seen Green Guide Michelin (Le Guide Vert) for outside Europe. It exists for NY and other major cities, but they are not very discriminating in their choice of restaurants which are lumped with tourist attractions. You will find more of a pot-pourri of restaurant suggestions from all over the spectrum.

[Pan]

It isn't easy to drive your car from Paris to Milwaukee.

[Pan]

As far as I know, I have only seen Green Guide Michelin (Le Guide Vert) for outside Europe. It exists for NY and other major cities, but they are not very discriminating in their choice of restaurants which are lumped with tourist attractions. You will find more of a pot-pourri of restaurant suggestions from all over the spectrum.

Thanks.

And is any of that on the www?

[Explorer]

I don't think it's on the www. What they seem to have is viamichelin.com which is European focused.

[Pan]

That's what I had figured, too.

I found Viamichelin useful in checking contact information and addresses of listed restaurants in Paris.

[jhlurie]

The Seattle P-I gives separate ratings for food, service, and ambience.  Is that a step in the right direction or wrong?

So does Zagat. + cost.

I often violently disagree with Zagat's actual rating, but the system itself isn't a bad starting point.

Regarding the issue of maintaining an absolute rating system, and yet recognizing the relative level of local quality, one method which occured to me would be to add an extra value to each rating which lists the score of the highest rated restaurant of THAT type in that region.

Lets say that you are rating an "Informal Mexican" restaurant (I would divide each cuisine into Formal and Informal). Very little Mexican food available in New Jersey is going to measure up to what's available in Texas, for example. Texas might be filled with Informal Mexican that on, lets say, a 100 point scale might almost always range between 70 and 100. The best Informal Mex place in ALL of NJ might be a 78. So if a new Mexican joint in NJ opens and scores a 75, I'd like to know that its nearest competition only got a 78. The score of the new place would be 75/78, at least until a place better than the joint that got the 78 opens in that area. So if you also cross reference it with a place in El Paso that scored a 94, you will be fully aware that in an ultimate sense the place in El Paso which grabbed the 94 is FAR better than the place in NJ with the 75, but that 75 is actually a pretty decent score in the local area.

Obviously this system isn't perfect, but its got some real advantages.

[Steve Plotnicki]

Obviously this system isn't perfect, but its got some real advantages.

Especially if you have like ten hours trying to figure out where to have dinner.

[Stone]

By height.