Restaurant Top 50 2004

[oakapple]

If Restaurant Magazine's ranking of the '50 Best Restaurants in the World' is not actually a ranking of the 50 best restaurants in the world, then why is it called the '50 Best Restaurants in the World'?

Because most people are willing to fill in the missing words "...subject to the limitations of the way we polled." It's patently obvious that a list of the "50 best" of anything not concretely measurable is merely one point of view. Human beings make such lists all the time. It is so well understood that this is just one point of view, that I don't think you can accuse them of fundamental dishonesty.

[Opson]

If Restaurant Magazine's ranking of the '50 Best Restaurants in the World' is not actually a ranking of the 50 best restaurants in the world...

...in your opinion.

Opson, the only way you will convince me is be providing me with your own list of the 50 best restaurants in the world, telling me how you arrived at your list and why your results and methodology are superior to that of Restaurant Magazine's. Then we might have something further to debate.

Thankfully, convincing you of irrelevant side issues doesn't have any bearing on the topic in hand.

However, if you genuinely believe that this ranking is definitive (this seems to be the case) then I challenge you to demonstrate that Restaurant Magazine's list is actually what it says it is.

[Andy Lynes]

Thankfully, convincing you of irrelevant side issues doesn't have any bearing on the topic in hand.

I was attempting to address dierctly the issue in hand i.e. the validity of the list. I am sanguine that we must agree to disagree on this and I'll therefore take my leave of this thread in the hope that it may flourish in my absense.

[thom]

Because most people are willing to fill in the missing words "...subject to the limitations of the way we polled." It's patently obvious that a list of the "50 best" of anything not concretely measurable is merely one point of view. Human beings make such lists all the time. It is so well understood that this is just one point of view, that I don't think you can accuse them of fundamental dishonesty.

Oakapple, you always beat me to the punch and explain it more eloquently than I would...

Basically, my reply is what Oakapple said. Seems perfectly clear to me. Opson, lets face it you aren't going to change your mind, and of course I know that I'm right so I'm running out of reasons to keep the debate going... You're not just doing this to get onto next years guest list are you?

As has been mentioned previously I still think a more interesting exercise is to look at the list, and in light of it's methodlogy (the opinions of 300 vastly experienced restaurateurs, chefs, critics and writers are not to be taken lightly), debate why they might have made the choices they have.

Cheers

Thom

[marcus]

As has been mentioned previously I still think a more interesting exercise is to look at the list, and in light of it's methodlogy (the opinions of 300 vastly experienced restaurateurs, chefs, critics and writers are not to be taken lightly), debate why they might have made the choices they have.

Well, we've been around on this before, but one of the main reasons that they made these choices is that more of them had eaten in certain restaurants than others. This had a major effect on the results. Is this a valid criterion??

[Opson]

Opson, lets face it you aren't going to change your mind, and of course I know that I'm right so I'm running out of reasons to keep the debate going... You're not just doing this to get onto next years guest list are you?

As has been mentioned previously I still think a more interesting exercise is to look at the list, and in light of it's methodlogy (the opinions of 300 vastly experienced restaurateurs, chefs, critics and writers are not to be taken lightly), debate why they might have made the choices they have.

Cheers

Thom

Thankyou for taking the time to not answer my question.

[ballast_regime]

How can one talk about the validity of such a list? There are something like 13 or 14 restaurants on the list that are in the UK, which means that Britain has approximately 25% of the world's best places to catch some great grub. Such a list is more about selling issues than about its putative merits or implications for weeding out good and bad restaurants. (Think about it: The French Laundry wasn't even open when this issue hit newsstands.)

Much peace,

Ian Lowe

ballast/regime

[robyn]

The original concept of Atelier was no reservations, and they followed this for the first few months that they were open, beginning just about a year ago.  The then changed there policy and started accepting reservations for the first seating for lunch and dinner only.  I personally made a dinner reservation at Atelier for the last Saturday in November 2003 for the first seating at 6:30.  As an aside, I found the food highly variable in quality, and every single person sitting on my side, the right hand counter, was English speaking.  I am personally confident that Atelier is not the 100th best restaurant in the world, let alone the 4th.

With regard to Robyn's post regarding Atelier not taking reservations, this is out of date.  I don't know how other people feel, but I believe that eGullet is best served by posters writing about what they know directly.  We can all Google the internet and find out for ourselves what's there.  Using this kind of information as the sole basis for a posting doesn't seem to me to be generally useful.

If a restaurant is supposed to be among the top 50 in the world (and charging accordingly) - I think *it* should have a web site which contains - among other things - accurate information about its reservations policy (so people won't have to rely on other web sites - or messages written by people they don't know). Robyn

[robyn]

Why didn't Trio make the list at all, I wonder?

It's not a "destination" restaurant - and it's in Chicago - which isn't a "destination" city.

I happen to love Chicago - and the eating there (and the architecture and the art!) - so don't blame it on me. Robyn

I have to strongly disagree with you here. Have you been to Trio? Have you been to Moto? Charlie Trotters? Chicago restaurants are much further advanced than any other major city in the world. I have just named 3 tasting menu only restaurants, there are 2 others........no other city in the world can even come close to that. Unfortunately, many diners take chicago for what it use to be. Look at it from an unbiased standpoint. For many years chicago has had a label as a "fast food town", I can tell you personally I know many chicago diners that are tired of that and support restaurants like Moto and Trio a lot. You cant come close to experiences like these anywhere else in the US. Chef Achatz & Cantu will lead Chicago to a very new level of dining. As far as Trio not being a destination restaurant - I think this makes me want to cry.

Whoa - don't shoot the messenger. I am a big fan of Chicago - and also happen to think that it has one of the best hotels in the world these days (The Peninsula). But just ask the people in this thread who don't live in the US whether they plan vacation trips to Chicago to wine/dine/look at art/etc. You don't have to convince *me* that they're missing something - you have to convince them! Robyn

[marcus]

If a restaurant is supposed to be among the top 50 in the world (and charging accordingly) - I think *it* should have a web site which contains - among other things - accurate information about its reservations policy (so people won't have to rely on other web sites - or messages written by people they don't know). Robyn

Many of the genuinely best restaurants in the world are quite small and low volume and haven't bothered with websites, Ambroisie is one example. They all take reservations, so you can call them and learn their specific policies. I'm not sure how individuals acquires a sense of entitlement that requires successful businesses to confirm to their desires and tells these businesses what they "should" do.

[marcus]

Hot off the New York board. Posted without further comment.

Hi all

Was just checking the latest top 50 in restaurant magazine. The top two rated restaurants in NY are Gramercy Tavern and Daniel. Which would you recommend of these two? I am a bit of a NY virgin when it comes to restaurants. I've been to One If By Land, Two if By Sea, which I loved, and Asia de Cuba.

Cheers'

DJO

[Gary Marshall]

The original concept of Atelier was no reservations, and they followed this for the first few months that they were open, beginning just about a year ago.  The then changed there policy and started accepting reservations for the first seating for lunch and dinner only.  I personally made a dinner reservation at Atelier for the last Saturday in November 2003 for the first seating at 6:30.  As an aside, I found the food highly variable in quality, and every single person sitting on my side, the right hand counter, was English speaking.  I am personally confident that Atelier is not the 100th best restaurant in the world, let alone the 4th.

With regard to Robyn's post regarding Atelier not taking reservations, this is out of date.  I don't know how other people feel, but I believe that eGullet is best served by posters writing about what they know directly.  We can all Google the internet and find out for ourselves what's there.  Using this kind of information as the sole basis for a posting doesn't seem to me to be generally useful.

If a restaurant is supposed to be among the top 50 in the world (and charging accordingly) - I think *it* should have a web site which contains - among other things - accurate information about its reservations policy (so people won't have to rely on other web sites - or messages written by people they don't know). Robyn

interpretation:

they should have a website so i don't uneccessarily and highly pedantically quote something about a restaurant i know zip about and subsequently be made to eat my words.

ah well, made me laugh this morning.

cheers

gary

[degusto]

Hot off the New York board. Posted without further comment.

Hi all

Was just checking the latest top 50 in restaurant magazine. The top two rated restaurants in NY are Gramercy Tavern and Daniel. Which would you recommend of these two? I am a bit of a NY virgin when it comes to restaurants. I've been to One If By Land, Two if By Sea, which I loved, and Asia de Cuba.

Cheers'

DJO

Oh please save this guy. Advice him immediately of the five pages with fine print, which the Restaurant Magazine has compiled in which all disclaimers, representations and definitions are lined out. He can obtain the fine print by addressing the Restaurant Magazine with a written request together with sufficient UK postage stamp for the delivery of the fine print.

I think that in that fine print within the section of definitions it says something along those lines:

DEFINITIONS

.

“The 50 best restaurants in the world” - Defined for the purpose of The List as a restaurant which is not necessarily one the 50 or even 1000 best restaurants in the world or even in Europe. If The List would have been compiled with some level of methodology it is likely that many of the restaurants appearing on The List would not have gained a single vote."

.

No seriously, I think he should go to Balthazar. I would have liked to see that place on the spot behind l'Atelier within the top five in the world. By the way, why did not Bibendum win? Bibendum, Aux Lyonnaise, The Lean Chicken, l'Atelier and Balthazar would have been the top five that would have made more sense.

[oakapple]

Was just checking the latest top 50 in restaurant magazine. The top two rated restaurants in NY are Gramercy Tavern and Daniel. Which would you recommend of these two?

The reader could just as easily have posted, "Was just checking the NY Times website. Asiate and Compass both received one star recently. Which would you recommend of these two?"

Translation: Yes, of course people pick up magazines and ask questions about what what they've read.

[marcus]

The reader could just as easily have posted, "Was just checking the NY Times website. Asiate and Compass both received one star recently. Which would you recommend of these two?"

Translation: Yes, of course people pick up magazines and ask questions about what what they've read.

This implicitly goes back to the flawed argument that since no rating system is perfect, all are equally deficient.

This person is looking for the best restaurant in NY. If he picked up the New York Times and searched for the 4 star restaurants he would have found:

ADNY

Bouley

Daniel

Jean Georges

Le Bernardin

This would have provided a much more useful and better starting point for his investigation. In fact, because of restaurant magazine's ratings, he's asking the wrong question, Daniel vs GT, because he thinks that he already knows the 2 best restaurants, but in fact he doesn't.

[argh_01]

The real downside is that gullible people might make real dining decisions based on this list.

quel désastre!

REAL dining decisions!! what a downside!!!

c'mon, marcus, it's just a magazine "most popular" list! a semi-serious measure of culinary zeitgeist, not some sort of live-or-die official ranking.

[Opson]

The real downside is that gullible people might make real dining decisions based on this list.

quel désastre!

REAL dining decisions!! what a downside!!!

c'mon, marcus, it's just a magazine "most popular" list! a semi-serious measure of culinary zeitgeist, not some sort of live-or-die official ranking.

Yes it isn't a live or die official ranking, it's just a publicity stunt. The problem is, however, that the results enter the public conciousness as if it were much more than an unscientific stunt. Indeed, if Heston Blumenthal refers to his own restaurant as being "second best in the world and first in Europe" with no reference to the invalidity of the poll, or even the pollsters, what hope is there for anyone else?

[Fat Guy]

In the front of the Zagat survey, there is a list of Most Popular restaurants. The list is prefaced by the explanation, "Each of our surveyors has been asked to name his or her five favorite restaurants. The following list reflects their choices, followed in parentheses by last year's ranking." In New York, the four-star restaurants emphatically do not dominate the top of this list. ADNY isn't even on it. Carmine's and Rosa Mexicano anchor the middle of the list. Apparently Union Square Cafe is number one. Gramercy Tavern and Daniel are numbers two and three. Craft is number 23. Balthazar is number 12.

Restaurant Magazine is doing essentially the same thing as Zagat, with a smaller and more knowledgeable group of survey participants, and a larger geographic coverage area. I would say that Restaurant Magazine's list, insofar as New York is concerned, is approximately as credible as Zagat's list, perhaps a bit more credible. In other words, it is no more ridiculous-seeming to me than Zagat.

Thom has done a good job from a PR perspective with this. Just yesterday, in my e-mail inbox, came a notice from the French Laundry's publicist regarding the award, addressed to all the top food media in the US, and also to me. The ceremony has developed into a James Beard Award-type of event, with sponsorship from Penfolds and other corporations and attendance by a who's-who of chefs. This is certainly an achievement.

I do feel that Thom's participation on this topic has been somewhat reductionistic: he should be taken to task for his snarky and ultimately empty "that's an opinion" line of argumentation. But I really don't see any other way for him to go. If he attempts to improve the methodology -- to compensate for attendance, geography, restaurant size, etc. -- he runs the risk of changing the list so radically that the UK will become a minor player and he'll have to start recruiting his chef-supporters from scratch.

Here at eGullet, where we have access to a worldwide pool of very experienced diners, we have long delayed getting into the restaurant ranking game. To us, it would be more important to do it right than to do it in a flashy, awards-ceremony and PR-oriented manner. Someday I hope we will have such rankings, but we won't do them until we feel we have come up with a highly credible methodology. At that point, it will be very interesting to compare results from people who care about methodology with those from people who hide from it.

[Gary Marshall]

I do feel that Thom's participation on this topic has been somewhat reductionistic: he should be taken to task for his snarky and ultimately empty "that's an opinion" line of argumentation. But I really don't see any other way for him to go. If he attempts to improve the methodology -- to compensate for attendance, geography, restaurant size, etc. -- he runs the risk of changing the list so radically that the UK will become a minor player and he'll have to start recruiting his chef-supporters from scratch.

but to be fair to thom, there's also another angle, as the organiser of what has become a successful and popular event, run by his employer, it would be professionally foolish to enter into a debate too deeply which essentially is ' the restaurant best 50 awards have no validity-discuss' especially if he wants to be further involved in the project.

also when his editor has already participated in, and is aware of the thread.

so i think it a little unfair, i'm sure he stands by the methodology, it makes sense to me, but even if he didn't it's unlikely he'd agree on a public website read by his employers don't you think?

cheers

gary

[Jonathan Day]

Let me pose the following question to this group: suppose that eGullet were to sponsor a restaurant rating or ranking or more formal reviewing system. Further suppose, contrary to fact, that resources were not an issue.

- What would you want the rating system to do that our forum-based reviews don't do right now, if anything?

- How would you go about rating or ranking restaurants? What criteria would you apply? What methodology would you use to apply those criteria?

- How would you deal with some of the issues already noted on this thread?

This isn't an attempt to design an eGullet system, more to shift the conversation from the critical to the constructive. Who knows, perhaps Restaurant Magazine could adopt our approach next year...

[marcus]

but to be fair to thom, there's also another angle, as the organiser of what has become a successful and popular event, run by his employer, it would be professionally foolish to enter into a debate too deeply which essentially is ' the restaurant best 50 awards have no validity-discuss' especially if he wants to be further involved in the project.

also when his editor has already participated in, and is aware of the thread.

so i think it a little unfair, i'm sure he stands by the methodology, it makes sense to me, but even if he didn't it's unlikely he'd agree on a public website read by his employers don't you think?

I think that the appropriate course of action if one doesn't want to forthrightly discuss a topic on a public website, is not to post at all. I don't think that stonewalling serves anyone's purpose.

[Winot]

In the front of the Zagat survey, there is a list of Most Popular restaurants.

And the Zagat survey is called that? "Most Popular"? That's the nub, isn't it? We wouldn't be having this discussion if the Restaurant Magazine's survey was "50 Most Popular Restaurants".

[Fat Guy]

Colloquially, however, everybody refers to that list as "The Zagat Top 50."

[oakapple]

Let me pose the following question to this group: suppose that eGullet were to sponsor a restaurant rating or ranking or more formal reviewing system. Further suppose, contrary to fact, that resources were not an issue.

If resources weren't an issue, I'd have a large, full-time, highly-qualified (and appropriately paid) staff, with an expense account about equal to the GDP of Monaco. As that isn't going to happen, I have a few more modest suggestions:

- What would you want the rating system to do that our forum-based reviews don't do right now, if anything?

eGullet's forum-based reviews are irregular, disjointed, non-systematic, and of widely varying quality. I enjoy them for what they are, but they aren't a rating system.

At the risk of sounding heretical, I think the Zagat guide is organized about the way you want. The problem with Zagat is the lack of rigor in the numbers themselves. Yet, Zagat has many desirable attributes:

• It shows each's restaurant's correct name, address, and telephone number
  
• It rates food, service, and decor separately
  
• It gives you a good idea of the price range
  
• Neighborhoods and cuisines are classified granularly
  
• There's a brief sample of multiple reviewers' comments
  
• It captures other useful indicia (accepts reservations, accepts credit cards, open Sundays, serves breakfast, serves lunch, romantic restaurant, great view, kid-friendly, etc.)
  

You can search by most of the above data elements. As a restaurant search engine, Zagat is actually pretty good, once you understand that the ratings themselves aren't terribly meaningful.

In the ideal eGullet ratings, I would also link to more detailed reviews, and also to the restaurant's own website (if it has one).

- How would you go about rating or ranking restaurants? What criteria would you apply? What methodology would you use to apply those criteria?

As noted above, I would rate food, service, and decor separately, and perhaps include an overall composite rating. This structure would satisfy those who care only about the food, and it would also satisfy those who are interested in the "overall experience."

I like one-to-four stars, rather than a numerical rating of one-to-twenty (Gayot) or one-to-thirty (Zagat). In a four-star rating, it's easier to establish and maintain rating criteria consistently. If you allow half-stars, that gives you nine gradations (including zero stars), and that's more than enough. Of course, implicit in this is that you try to define the star criteria--something that neither the NYTimes or Michelin has done very well.

I think the system needs to formally recognize the restaurant's intent (or apparent intent). A two-star restaurant can be a two-star concept executed well, a three-star concept with a few problems, or a four-star concept that's failing abysmally. Someone searching for a two-star restaurant probably wants the first kind, not the second or the third.

In my view, you need separate surveys for each city or region that you want to cover. For restaurants in Spain and Turkey to be on the same list is really useless, except as a lively topic of conversation. I don't think there is any economically-feasible methodology that would allow you to mix results from different geographies with any rigor, because the reviewers will have too little in common.

Within a city or region, the main question is: are you trying to rate a broad range (like Zagat), or only the high-end restaurants (like Michelin)? In a major international city, there are thousands of restaurants. If you're covering only the top, say, 50-100 of them, it shouldn't be too difficult to assemble a group of qualified reviewers, who have eaten at enough of those top restaurants to have relevant experience in common.

If you're trying to cover the area more broadly, then I think the reviewers need to be qualified by "category." For instance, somebody who regularly eats in noodle shops and pizza parlors might be extraordinarily well qualified to vote in those categories. But if that same person splurges one day at Jean-Georges, that does not qualify her to vote for Jean-Georges, when she hasn't tried any of the other restaurants in its class.

- How would you deal with some of the issues already noted on this thread?

Do separate surveys in each city or region where there is sufficient critical mass to get enough qualified voters. Put restaurants in categories, and then rank by stars. Don't attempt to impose a strict rank ordering, as no methodology will reliably differentiate #18 from #22.

Marcus was concerned that restaurants could be highly rated simply because they are oft-visited. As he put it:

This is a very important point. This introduces a significant popularity factor into the ratings, all other things being equal, the restaurants that have been visited by more of the judges, will score higher. This causes a major systematic error in the results. This conceivably could have been compensated for by a normalization process where every judge listed the restaurants that they had and hadn't visited, and the scores of the less visited restaurants were appropriately increased.

Normalization introduces the opposite problem: a seldom-visited restaurant that hasn't proved itself across a wide spectrum of voters might get rated more highly than it deserves. I wouldn't try to normalize. A restaurant needs to get at least N votes. If it has that many, and the average is 3 stars, then it's a 3-star restaurant. Zagat has a little symbol that it uses to designate restaurants that have had a small number of votes, to signal that the rating might be less reliable. Zagat's notation for restaurants garnering mixed reviews is also useful (i.e., where the range of votes wide, suggesting uneven food/service).

Lastly, I would suggest moving this branch of the discussion to a new thread, as I think many people who would contribute are tired of the heated temperature of the Restaurant Magazine debate.

[chengb02]

some thoughts...I totally agree with something Lisa J said, these lists are created for a very distinct purpose, to gain publicity and sell magazines. Look at all the conversation that has went on about it here. I'm sure many of us have went to the website a number of times (or bought/read the magazine). All magazines do this because its a great way to get people talking about your magazine. Those people who base where they eat solely on this list are doing so more to be able to say "I've been there" rather than for the food. That is not to say if somebody went to any of the places in the top 10 they'd be disappointed. The list is published by a UK magazine, so we should expect it to be UK or Euro-centric. It does include restaurants from all over the world, but it does so here and there. I agree that Trio belongs on the list, but Charlie Trotter's is a fair representative from Chicago. I'm sure people in LA feel there are a number of restaurants that deserve to be on the list instead of Spago, but LA also only got one representative. I'm sort of surprised that there is nothing from Bangkok, Singapore, Shanghai, or Tokyo. There is Felix from HK, I'd put a few other places above it, but its not a flawed choice...

So is there any restaurant that blatantly doesn't deserve to be on the list? Maybe one or two, but the list isn't perfect and all places will provide an excellent meal. Just like listing music or beautiful people or anything else, how can you really say one food tastes better than another, especially when looking at different cuisines...