Let's create a better Restaurant Rating System

[Pan]

JH, you're really speaking my language, and particularly here:

I'm still a bit dubious about how you would determine "the 50 or so most important new ones". This breaks down on the lower end, and its my contention that this is where most current systems already dissapoint.

Exactly. But I think that there are enough eGulleteers interested in the lower end and frequenting inexpensive and moderately inexpensive Arab, Chinese, Malaysian, Korean, Italian, and various other styles of food, and we're geographically distributed through, for example, the New York area better than, for example, Zagat respondants. I always find it totally pathetic how Manhattan- and, particularly, Upper West Side(!)-centric their respondants are. I mean, the Upper West Side as a mecca of good food? Give me a break!

[Wilfrid]

What I took away from Ron Johnson's thread is that there's really very little disagreement about the top two tiers in New York. I think we could go through all kinds of hoops, and come out with the answer that ADNY is in the top six and that Craft isn't, but it's in the next tier down.

Another way to look at this would be: What's missing from the existing rankings? What's wrong with them? I agree that one gap that could usefully be filled - still for NYC - would be something like a best two or three lower cost restaurants, in a variety of categories, per borough. Like Steve, I think there's limited value in knowing what the best ten burger joints or Cantonese restaurants in Queen's are, but I'd surely like to know the best two or three.

We have been doing a kind of haphazard version of this for Italian food on this old thread.

[Steve Plotnicki]

While its true that I absolutely wouldn't care to see ranking or ratings of every little hamburger joint on every corner, I WOULD like to see a system that realistically included the best of them somehow.

You have all now just run up on the hard part of the problem. How do you codify different classes of restaurants? Because for example, having a pastrami sandwich at Katz's is one of the 50 most important meals you need to eat in NYC. In fact it might be in the top 25, or even the top 10. But it is not one of the best meals you can have if you you use any type of formal definition of the word meal. It is a casual meal, and in a category of its own, separate and apart from fine dining. But the reason guide books include it is because the food is so good that it attracts all classes of diners. So the guide books report on this phenomenon. But a guide book that covers fine dining, which is for an exclusive class, needs to take a sharp left turn in order to start covering ethnic restaurants that are not patronized by the same people who patronize places like Gramercy Tavern.

[jhlurie]

I don't know Steve... it certainly is a puzzle, but its one I hope we can solve. I do like your distinction between "important meals" and what is usually used to review or rank restaurants. BOTH of these things need to be catalogued somehow.

We all know, however, that the dividing line between fine dining and ethnic eats isn't as hard and fast as all that though. There are degrees of seperation.

Edited March 4, 2003 by jhlurie (log)

[Fat Guy]

50 most important meals you need to eat in NYC

That's a list one could make. I don't see what the big deal is. How is this such a big puzzle? You figure out the lists people want and will use, and you create them.

[Steve Plotnicki]

Important meals to whom?

[Fat Guy]

having a pastrami sandwich at Katz's is one of the 50 most important meals you need to eat in NYC

What did you mean when you said it?

[Wilfrid]

That's a list one could make. I don't see what the big deal is. How is this such a big puzzle? You figure out the lists people want and will use, and you create them.

Part of which must involve figuring out why people don't want any of the lists currently available. I'm still unclear about that.

What exactly is being suggested here that Zagat doesn't do?

[Fat Guy]

What exactly is being suggested here that Zagat doesn't do?

What does Michelin do that Zagat doesn't do? That's the first level on which I'd approach this. The big difference in my mind -- the one that makes Michelin credible and Zagat a completely useless failure and joke -- is Michelin's qualified "inspectors" championing excellence instead of Zagat's undifferentiated group reflecting amalgamated, average middle-market tastes.

In addition, I'd propose to offer real-time updates, listings broken down by price/fanciness level in each category (and, therefore, freedom from the apples-and-oranges aggregated scoring system), rankings instead of ratings, aggregated lists that include specific dish recommendations, and a few other things I'm still thinking of.

[Steve Plotnicki]

The big difference in my mind -- the one that makes Michelin credible and Zagat a completely useless failure and joke -- is Michelin's qualified "inspectors" championing excellence instead of Zagat's undifferentiated group reflecting amalgamated, average middle-market tastes.

Hooray for Plotnickiism.

[Wilfrid]

I agree the outcome achieved by Zagat is ropey, because of who reviews and how they do it (no obligation to visit the restaurants). I was wondering how we could improve the structure: Zagat does rank (and the rankings aren't absurd overall), it does to some extent break down cuisines by styles and fanciness. I guess this approach could be developed further.

[Fat Guy]

Yes, though I think we should make clear that Zagat did not invent or even contribute much to that approach.

[Explorer]

I think we're getting somewhere now...

The only way that the outside world will notice is if we truly end-up with a breakthough system that users can associate with; something really different from existing systems that takes advantage of the unique capabilities of the Web: 1) interactivity (I can sort/resort); 2) personalization (according to my preferences); 3) real-time (knowing as soon as it happens).

Somewhere between the single-handed nature of Michelin/G-M inspectors or newspaper food critics AND the popular vote system lies the new eGullet approach: a networked panel of discriminating reviewers that can be trusted as beacons of accuracy for describing restaurants worth visiting.

But what is our system? Zagat has the 30 point system, Michelin- the stars, G-M - the 20 points. I know some members are against ratings, but another ranking on its own may not enough to create a buzz. Rankings within "smart" categories like the ones being suggested here? Perhaps; but am not sure it's enough either. Is it the real-time aspect? Perhaps.

I think we need to push a little harder on asking ourselves: how different, how much better will this be? Am looking for a short sound-bite, a sentence that we can all agree upon that encapsulates the vision.

[Fat Guy]

I think being anti-ratings should be part of the manifesto, just like weighing dry ingredients should be part of the manifesto for our recipe database. The lack of stars and scores shouldn't be something we have to make excuses for, it should be a major selling point. I'm not immediately interested in mass-market appeal. The public is so screwed up about all these issues, our mission should be to straighten it all out.

[Explorer]

The lack of stars and scores shouldn't be something we have to make excuses for, it should be a major selling point.

Great, I agree ; I was pushing a bit more to be sure.

So, we just articulated the first principle, I think. It's simple and clear.

Principle #2:

I'm not immediately interested in mass-market appeal

I agree with that too; this isn't a list for the masses; BUT the masses may look at it and follow it because it's the visionary list that everybody wants to be in on. So, we captured our primary and secondary market.

Principle #3:

It's based on real-time reporting by a network of experts and updated every xxx...(not sure: weekly, daily, monthly).

More simple statements....The simpler, the easier we can agree on them. Note these are our internal brainstorming statements.

[Fat Guy]

Principle #3:

It's based on real-time reporting by a network of experts and updated . . .

Continuously. As soon as an "inspector" enters a meal report, it is factored into the database.

[Steve Plotnicki]

The only way that the outside world will notice is if we truly end-up with a breakthough system that users can associate with; something really different from existing systems that takes advantage of the unique capabilities of the Web:

You will always end up being stuck between a class based system and an egalitarian one. Diners break down into those two categories no matter how hard you want to believe otherwise. You are better off going with one or the other. Michelin and Gaullt Millau are great examples of the class model. Zagat is a great example of the egalitarian, populist model. And it isn't that you can't find inspectors who do not understand both of these aspects of dining, Your problem comes in how the codification system works. When are you recommending a restaurant based on class and when are you recommending just based on the quality of cuisine. In the upper middle you recommend both by default. But how do you succinctly describe that Congee Village revolves around a different principal?

[Fat Guy]

But how do you succinctly describe that Congee Village revolves around a different principal?

By putting it on a different list.

[Steve Plotnicki]

That defeats the purpose. What would be groundbreaking is if you were able to codify them all on the same list and signal their relative importance to diners.

[Fat Guy]

What would be groundbreaking is if you were able to codify them all on the same list and signal their relative importance to diners.

I thought you said that's impossible.

[Fat Guy]

That defeats the purpose.

Why?

[Steve Plotnicki]

It's impossible because one category of people doesn't dine in restaurants in the other category. To people who are in the ethnic/cheap eats category, they are not happy seeing Congee Village and Sripaphai coming in at 35th place, after 34 versions of Daniel and Gramercry Tavern. It's the same inference of inferiority that besets wines that Robert Parker scores at 89 or 90. That they are good wines is irrelevent. What jumps out at people is the huge increment between those wines and ones that score 100. This issue, even though people like you and I might be immune to this, turns people off because it makes the list seem like it is not directed towards them. And I thought the purpose was to make one big list that everyone felt was directed towards them without succumbing to Zagat style populism?

[Wilfrid]

I think the idea is lots of lists, though. Am I right? Sticking with NYC, I'd love lists, divided by borough, which gave me the best 3 cheap Chinese, the best 3 expensive Chinese (if any), the best 3 pizzas, the best 3 burgers, and so on.

If one resists lists of 30 restaurants, then stars and points, and arguably rankings are unnecessary. If one wants a long list, then even if one does without stars and points, ranking would be essential.

[Fat Guy]

I think the idea is lots of lists, though.  Am I right?

Lots of lists, yes, but not a dense presentation. For the dabbler, there can be just a few key lists that go shallow on details but still present credible top-level summary information. And for the serious readers, there can be much more accurate breakdowns that tell the whole story.

[Explorer]

There's a difference between ad-hoc lists generated by individual queries and category-specific-lists that are based on our own classification. For e.g., cutting it by borough becomes a technical/query thing; if you do that it will give you your own list which will include different classifications.

What I am hearing is that we want a "best-in-class" list; so how many "classes" do we want to identify?

For e.g. "my" list based on my query for region X or Borough X:

1. Rest 1 from Category A

2. Rest 2 from Category A

3. Rest 3 from Category B

4. Rest 4 from Category B

5. Res 5 from Category C

6. etc..

So if you say, I feel like eating in Category C, so I pick Rest #5 and double-click on it, it takes me 1) formal reviews on it with more detail, 2) discussions on it, etc...

Or if you want all of the Category A, then you get them regardless of where they are.