Let's create a better Restaurant Rating System

[Wilfrid]

I think I get it. Yes, a long list of classes, but not necessarily a long list of restaurants is my vote. I agree you can customize your searching to identify restaurants in a particular borough; my point was that, if we did this for NYC, I think I'd prefer a list of the best three pizza joints in Manhattan, the Bronx, Queens and Brooklyn (Staten Island if necessary), totalling - let's see - 15 restaurants - rather than a list of the best 15 regardless of which borough they're in. Maybe just me.

[Fat Guy]

I'm not interested in creating a restaurant database ala CitySearch or Zagat. It's a very expensive undertaking and in the end nets, at least in Zagat's case, little more than a glorified address book. At least CitySearch offers editorial reviews by real writers. But I don't see us going in that direction -- I see us as more akin to Michelin, but with a modern, new-media approach and and an anti-star attitude.

What I'm talking about is the best of the best. How many restaurants does Zagat cover? I think it's about 2000 for New York City. That's just unnecessary. I'd say that 80-90% of the restaurants listed in Zagat New York would never make it onto any top-10 list, save for "top 10 restaurants that don't deserve to be on any top 10 list." And that's where the grain of truth in Plotnicki's statements is to be found: if you're going to be more than a fancy white pages, there's no reason to list any restaurant that isn't among the very best representatives in its category, whatever that category may be. Some categories are going to be larger than others, but there has to be a minimum standard. If there is no really good mid-priced restaurant of ethnicity X in New York City, we need to be willing to say that no restaurant in that category meets our standard, and therefore the category is empty.

In terms of price and ethnicity, then, no custom searches are necessary. Those are all broken down by pre-set list. The only needed database functions would be a filter by neighborhood and a filter by restaurant name. Leave it to Zagat to tell people where they can find interesting bathrooms. Leave it to CitySearch to let people figure out who has the best outdoor seating areas. We should focus on conveying the knowledge of the greatest assemblage of knowledgeable gourmets in the world. We should do it first for New York, then for London, and then for every city where we can assemble a compelling demographic. Fully automated with minimum need for editorial intervention save for the startup effort and the occasional tweak by a committee of regional experts. A lean and mean system that provides the information serious eaters really want -- not all this fluff and crap being provided by the current big players.

[Explorer]

Bingo! Best-of-the Best, best-in-class. I vote for that approach.

I think you have articulated that vision pretty well.

[Steve Plotnicki]

Bingo! Best-of-the Best, best-in-class. I vote for that approach.

I think you have articulated that vision pretty well.

But this is what most guide books already do. They list 25-50 places in the French or Upper Middle Category, a bunch of the best steak houses and unique NYC eateries like the Oyster Bar or Rao's or John's Pizza, and then 2-3 of the best restaurants in every ethnic category. That fills an entire book. What is so new and so different in what you are proposing? Even Zagat has a list of the top five restaurants by category and that covers many ethnic categories. So what exactly is it that this is going to add to the mix?

[Fat Guy]

There's nothing new about ranking or rating restaurants. What can be new is doing it better: a more knowledgeable class of people doing the ratings, and an organizational scheme that actually reflects the reality of the restaurant landscape in a given city. That's all.

Can you name the "most guide books" you're talking about? What are you saying fills a whole book? The lists in Zagat, for example, fill just a few pages. Zagat's lists, moreover, are undifferentiated by price/hauteness category -- that's the major organizational problem (quite aside from the problem of uninformed survey participants): save for "French" and "French bistro" everything else is monolithic. What possible rationale could there be for ranking Vong and Planet Thailand on the same list? What consumer intends to choose between those two restaurants for a given meal out?

[g.johnson]

If it hasn't already come up, I'd suggest that the reviewers also categorize restaurants visited by rank rather than score them. That way avoids grade inflation, prevents eccentric scoring (e.g., someone gives Gramercy Tavern a 10 and every other restaurant in the category less than 4) and is immune to inter-rater variation.

[Steve Plotnicki]

I'm talking about your typical guide to dining published in book format, usually by a newspaper encapsulating their reviews of a city, or just a restaurant reviewer's book. Exactly the thing I might buy when I go out of town because I want a more in depth assessment of where to eat. Here's one I bought when I was in Amsterdam in January, IENS Independant Restaurant Index published in both Dutch and English. Most large cities have a version of this type of book.

The problem with these types of books is always the same. You need a way to calibrate your palate and your environment expectations against what they write and they only way to do that is to physically sample their taste. Good guide books have figured out a way through the narrative to express their taste clearly. Like the first time I bought a Gault Millau Guide in English. I suspected that their 12 and 13 point ratings where they admired how they grilled meats and fish would be the type of places I would like and they were.

So the rock and a hard place is, you need a narrative (which can be expressed numerically as well,) or editorial slant, that permeates the guide in a way that communicates a clear picture. But the problem is that in order to grasp what the narrative is saying, readers need to make an investment of time to read ththrough a substantial portion of the data to learn what the narrative is. What you would like to do is to condense that process. Good luck. I don't think it can be done. You might be able to organize it in a way that helps people get through it a little more quickly. But I'm not sure how much value that has over any other method of doing it.

[Explorer]

What can be new is doing it better: a more knowledgeable class of people doing the ratings, and an organizational scheme that actually reflects the reality of the restaurant landscape in a given city. That's all.

The above quote captures it. We have to think that 1) our small "network of expertise" is better than popular voting and better than 1-2 expert reviewers, 2) we'll be the only real real-time guide that's as accurate as it's current. Nobody does that.

A bit like the Relais et Chateaux approach. I have never been dissapointed with their selections in several countries; it's a rather small list, but very consistent. Some of their properties although not on Michelin are as good as a 1 or 2 star.

[Pan]

If there is no really good mid-priced restaurant of ethnicity X in New York City, we need to be willing to say that no restaurant in that category meets our standard, and therefore the category is empty.

Yeah, like Burmese food in New York. Unless someone knows something I don't, it's been a long time since there was a decent Burmese restaurant here.

[Fat Guy]

Also, the Canadian restaurant scene has really suffered since the Royal Canadian Pancake House closed. Not to mention the Polynesian restaurant situation is also, at present, a disaster.

[Steve Plotnicki]

Yes where's Trader Vic's when you need it.

[it aint easy being cheesy]

http://www.mobiltravelguide.com/index.jsp?...rating_criteria

Mobil and Triple A both have set rules that they judge by. It seems to make it more organized. I don't think its the end all but I must say that while some of the guidelines are arguable at least they are not random.

[Pan]

Getting back to another idea, I think that a "top-3" list would be useful, but I don't agree with Steve Plotnicki or anyone else who thinks that a "top-5" or "top-10" list or anything in between would be less useful. It's really a question of how many places you want to list, and I don't really have a problem with it either way, except that if the top 3 inexpensive Chinese places are all in Flushing and I'm in Manhattan and want to go to an inexpensive Chinese restaurant, I have to go to #4 (or something) on the list. But if there's no #4....Yes, I know this is partly remedied by dividing the "top-" listings by borough, but there's still the point that sometimes, people want convenience and quality to go together more. It's not always possible, as the thread on Chinese restaurants on the Upper East Side of Manhattan seemed to show, but it's more likely with a "top-10" than a "top-3" list, I think.

[Wilfrid]

Reviving this thread, is anyone still enthusiastic about the idea? If we start with New York, I'm willing to give some thought to drafting the categories. As for how many restaurants to list in each, then - if we do divide it by borough - I strongly feel it should be fewer than ten. I mean, who needs to know what the ninth best place in Manhattan is for dim sum? They can't get into the first eight? But I'm not wedded to three either. Maybe top five?

What do people think?

[g.johnson]

Depends on restaurant type, I think. I don't think many would want to go the more than one dim sum restaurant but they may want to go to several of the 3/4 star places so there should be more of those.

[Wilfrid]

Ah, I wasn't thinking of stars. I was thinking along the lines of multiple categories (as I think Fat Bloke was suggesting): Best Upscale Italian, Best Moderate Italian, Best Inexpensive Italian, Best Pizza, Best Dim Sum, Best fish and chips, Best takeaway sushi, and so on and on. Rather than a big raft of undifferentiated two star or 15/20 places like some guides.

[g.johnson]

I know, perhaps I wasn't clear. I think there should be a larger number of entries in categories covering "destination restaurants" (ugh) than ethnic restaurants, say. Ron Johnson, for example, appears to be doing at least two nose-bleedingly expensive French restaurants on his trip. I doubt he'll be doing more than one dim sum.

[Wilfrid]

Perhaps. But ten? How about top five, he said repeatedly.

[g.johnson]

Depends on how tightly defined the categories are. If your doing "Top French", "Top New American", etc., etc., five is about right. If it's "Top Top Restaurants" you might want to make it larger. It doesn't have to be rigid, though. "Top French" would have to be six to include all four stars, moreif you want to add Atelier, Le Cirque, etc. "Top New American" will be even harder to confine to five.

[Wilfrid]

Does anyone else see excitement, merit and novelty in making the lists quite limited? So, for example, a quite well known restaurant may just not make the cut in its particular category. I was sort of hoping that eGullet lists might not simply reflect the received wisdom of Zagat, NYT, et al. For example if - and I only say if - Gramercy Tavern didn't make top five upscale American restaurants, we'd be saying something. The membership (to the extent they participate) would be making an interesting statement.

Keeping this thread up here, I'm hoping some affiliates will chip in with a view as well. Ultimately, if we're going to have eGullet lists, the affiliates will have the final say on the structure and categories. So I don't want to do a bunch of drafting unless there's some interest.

Edited March 18, 2003 by Wilfrid (log)

[g.johnson]

You have greater faith in eGullet radicalism than I do.

[Wilfrid]

I dunno. If it was real-time voting, without self-appointed experts fiddling with the results, we might get some quirky and opinionated lists.

[SobaAddict70]

Bumping this thread up in honor of the "NYT Correction/Spice Market" thread in Food and Media.

Got any updated suggestions?

Soba

[hathor]

It's impossible because one category of people doesn't dine in restaurants in the other category.

What? Are you saying NO ONE who dines at Daniel or Jean Georges ever dines at the local Spaghetti Palace? These days, we mix our Chanel with the Gap.

[hathor]

I'm not ready to join the discussion on the framework of how/why a restaurant guide system would be valuable, but trying to find some reliable restaurant beta is how I came upon eGullet.

I would be happy if there was a restaurant forum or data base for eGullet members. It seems as if there are a lot of threads: 2 nights in Paris, 1 week in San Francisco....etc. The seach function on eGullet is a bit frustrating, so I would like to have one location, where if I'm going to Paris, I can find lots of opinions.

So, lets say its organized loosely as:

Paris

High End

Mid Range

Cheap Eats

When an eGullet member goes to a restaurant that he feels is mention worthy, he can enter the reveiw under that heading. It would be easier to locate the info when you are looking for it.

Part of the beauty of eGullet is the broad range of opinions that are expressed, and that was what initially attracted me to the web site. I would hate to lose that personality in a more impersonal ranking system.

Perhaps, after the 'data base' was up and running for awhile, then you could do a ranking with 'inspectors' culling and evaluating the reviews. A more organic approach to the guide, perhaps. Let it evolve and see what works.