Bruni and Beyond: Reviewing (2008)

[oakapple]

Sam, you can find your demonstration of the difference between anonymous reviewers and Fat Guy's "100% non-anonymous" reviewers right here.

What, exactly, do you think this demonstrates? It demonstrates that one non-anonymous reviewer (Bruni) gave Kobe Club a poor review whereas, according to Chodorow, three other equally non-anonymous reviewers (Greene, Lape, Mariani) "loved it." It's also possible that at least Green and Lape were "anonymous" when they dined there.

I don't know about Lape, but Greene claims to observe Bruni's brand of anonymity—i.e., she doesn't call attention to herself, but realizes (and regularly acknowledges) that she's likely to be recognized.

Surely you are not so naive to suppose that Bruni is not immediately recognized at a Chodorow restaurant?!

←

Given the sloppiness I've experienced at those restaurants, I could quite imagine that Bruni has paid them unrecognized visits, though certainly not a whole string of them.

Dave, you propose any number of justifications for presumed, if not actual anonymity. Many of them sound quite reasonable on paper.  But "on paper" isn't my bottom line.  My bottom line is the question: can we say for sure that presumed anonymity leads to better, more accurate and informative reviews?  I just don't see that it does.

Part of the problem with your question is that practically all of the front-line reviewers in this town practice some version of the Bruni system. Actually, I can't think of a notable exception.

The only thing we can say is that one regularly reads accounts of critic meals in which something-or-other occurred that is unlikely to have occurred if the critic were recognized. It's rare that one reads a whole review full of them, as we did with Ago, simply because most restaurants don't suck that badly while being prominent enough to merit a review.

Now, part of this may be due to the fact that we haven't exactly been blessed with the greatest reviewers at the NY Times for some number of years now.

I think Bruni's limitations are his limitations, whether he dines anonymously or not.

But, unless we can demonstrate that the "notionally anonymous" guys are turning out better and more accurate work than the non-anonymous guys, for me the argument in favor of presumed anonymity must fail.

←

Among regular reviewers in this town, Restaurant Girl and Andrea Strong are the ones making the case for non-anonymous reviewers, and I don't know anyone who thinks we need more reviews like theirs.

[slkinsey]

The story of Bob Lape I already linked to in my previous post, but there it is again if you couldn't find it.

You'll have to excuse me if I don't exactly take innuendo from Page Six and Gawker as accepted fact.

Edited June 16, 2008 by slkinsey (log)

[Dave H]

I don't think it's necessary for a reviewer to make a reservation in his own name, nor to announce himself upon entry into the restaurant to give up the myth of anonymity.

Do you really think John Mariani has more credibility than the New York Times? Or do you not care because the Times's extra credibility makes it harder for non-Times reviewers to do their jobs?

I think that David Rosengarten, as non-anonymous a reviewer as you are likely to find, has more credibility than the NY Times -- certainly under the last few reviewers.

←

Certainly David Rosengarten has plenty of credibility. Not being a subscriber I'm unfamiliar with the protocol he uses for "Rosengarten Report" reviews. Is he non-anonymous in the sense that he has his picture published and appears on TV--i.e. in the sense that Alan Richman is "non-anonymous"? In that case, I don't think the difference much matters; yes, it's silly (and PR theatrics) for Frank Bruni to only appear on TV with his face digitally blurred, but the visibility and relative importance of the NYT review is such that it really does make sense to try to make the average restaurant staff less likely recognize Bruni as opposed to more reviewers for more obscure outlets. (Whether these attempts have any effect is another question.)

If the Rosengarten Report is more in the line of magazine feature journalism--i.e. he travels to a particular restaurant, pre-announced, and writes the equivalent of a feature story on it--like when a particular restaurant is the cover story on Saveur, or featured on Tony Bourdain's show--then that sort of thing is valuable, and still relies on Rosengarten's credibility as a gastronome and cook and journalist and author, but it is in a different category from what Bruni does, or Richman does, or Michelin does. (The problem with John Mariani IMO is that he blurs these categories; he makes claims to objectivity, and to rank restaurants against each other--"The Best X New Restaurants", etc.--while following the protocols of feature journalism.)

If it's something else, then depending on the details perhaps I do have a problem with the protocol he uses for his reviews. It's important to point out that while a particular protocol could work for David Rosengarten, it might not work as well for an institutional position like NYT reviewer.

[Dave H]

The story of Bob Lape I already linked to in my previous post, but there it is again if you couldn't find it.

You'll have to excuse me if I don't exactly take innuendo from Page Six and Gawker as accepted fact.

←

The innuendo comes from Danny Meyer. Gawker just did the minimal legwork required to work out who Meyer was talking about.

[slkinsey]

But, unless we can demonstrate that the "notionally anonymous" guys are turning out better and more accurate work than the non-anonymous guys, for me the argument in favor of presumed anonymity must fail.

←

Among regular reviewers in this town, Restaurant Girl and Andrea Strong are the ones making the case for non-anonymous reviewers, and I don't know anyone who thinks we need more reviews like theirs.

That's a bit of a cheap shot. I could easily say, "among regular reviewers in this town, Frank Bruni and Adam Platt are the ones making the case for non-anonymous reviewers, and I don't know anyone who thinks we need more reviews like theirs."

Whose reviews would you find more valuable: Frank Bruni or David Rosengarten? (Dave: Rosengarten used to be the restaurant critic for "Gourmet.")

Edited June 16, 2008 by slkinsey (log)

[Dave H]

The story of Bob Lape I already linked to in my previous post, but there it is again if you couldn't find it.

You'll have to excuse me if I don't exactly take innuendo from Page Six and Gawker as accepted fact.

←

The innuendo comes from Danny Meyer. Gawker just did the minimal legwork required to work out who Meyer was talking about.

←

Edit: it's not innuendo. It is a baldfaced accusation of soliciting and rewarding bribes. Well, I suppose the accusation that everyone else in the restaurant industry knows and plays to Lape's M.O. qualifies as hearsay.

Further edit: and this isn't some off-the-record comment Danny Meyer made that was overheard by a gossip rag. This is something Danny Meyer published in his book.

Edited June 16, 2008 by Dave H (log)

[Nathan]

The story of Bob Lape I already linked to in my previous post, but there it is again if you couldn't find it.

You'll have to excuse me if I don't exactly take innuendo from Page Six and Gawker as accepted fact.

←

they source.

[Fat Guy]

A couple of these points are strawmen: "destructive game"? "toxic myth"? "living a contradiction"? Strong words, those. "Destructive" implies "destruction", and I don't see any. Since all critics acknowledge that they're frequently recognized, there really isn't any myth.

←

I'm pretty sure what you're trying to say is that my arguments are exaggerated, since the term "straw man" wouldn't make sense in the context of what you're saying. But I the game is destructive, i.e, a net negative for the quality of dining in the city. The myth that all restaurants are out to get us is toxic,in that it pits consumers against restaurants rather than helping them to build good relationships with restaurants. And there's a very clear contradiction: while, when queried directly, Frank Bruni will admit he's often recognized, the New York Times and Frank Bruni are nonetheless quite happy to have the average reader believe he's far more anonymous than he is. Do you really think the average reader of a New York Times review knows that the Times critic is recognized 80% of the time? Of course not. The Times has invested decades in fostering the anonymity myth, from Ruth Reichl's disguises (which didn't fool anyone) to persistent underreporting of the critic being recognized.

[Fat Guy]

Fat Guy seems to be arguing as if there's no difference between when a pseudo-anonymous reviewer like Bruni gets recognized in the middle of service and the sort of pre-announced fully-comped let-the-kitchen-cook-for-me "reviews" of a John Mariani. (Or of a "Travel & Leisure" type journalist, although at least those don't typically claim objectivity. And yes, I know that GQ pays for some of Mariani's meals, but he extracts others and indeed entire vacations from restaurants and chambers of commerce, etc.)

←

John Mariani doesn't write for GQ, as far as I know. He writes for Esquire, as well as in his own newsletter and in various other outlets (e.g., Bloomberg, where he's the wine columnist).

Have you ever read John Mariani's restaurant writing? Mariani knows so much more about food and restaurants than Frank Bruni it's shocking. You can get this from reading just a few issues of his newsletter, "Virtual Gourmet," which I read every week without fail.

Do I think Mariani is perfect? No. Far from it. Do I agree with all his conclusions? No, in fact I'm just working on an email to him about how I think he missed the boat on Benoit (he does, by the way, write plenty negative in his newsletter, for example he was not terribly kind to Benoit). But do I get more out of his writing than I do out of Frank Bruni's? Absolutely. Not only do I find his restaurant writing more insightful (though it usually lacks the multiple-visits dimension of the Times reviews), but also he is a more well-rounded authority, giving equal time to wine, travel, books, etc.

[Leonard Kim]

Perhaps I'm being cynical, but I always viewed Reichl 's followup review of Le Cirque as an example where she was probably "gamed." When Grimes reviewed it himself less than four years later, he noted, "I was spotted on my first visit (once again, the restaurant upheld its noble tradition of immediately recognizing critics)."

Reichl's Tavern on the Green review would, I'd guess, be a reasonable example of why she, at least, thought anonymity can be useful, though I guess this would be another case where a counter-argument might run, "everybody knows that about that place anyway."

If one is only looking at relative rankings, then I understand where anonymity can be dispensed with, and the playing field should be as level as possible.

But we all insist a review's value shouldn't lie just in the rating, but in the text. So far as the function of a review is for a critic to do advance work so that the reader can decide whether a place is worth an investment of time and money, I'd think most readers have an expectation that what's described in a review is, to some extent, representative of what they can expect to experience. With that goal in mind, the pretense of anonymity and occasional true anonymity is probably pointless and irrelevant in most cases but may be useful occasionally enough to be worth it, as long as neither critic (as in the first review) nor restaurant (as in the second review, where recognizing the critic ultimately lead to a worse, more damning review) are being naive about it.

If there's a disconnect between what's written and what's experienced, then readers complain, and the newspaper's or critic's credibility may suffer (or not.) Past NY Times critics would occasionally mention dissenting readers (as in the Miller Four Seasons review, which I was actually going to cite, but FG beat me to it. I have to admit I was going with Marc's interpretation.) Admittedly I can't think of an example of a critic admitting he/she might have been gamed. Sheraton's explanation was often along the lines of, service gets worse because restaurants can't cope with the upsurge of business following a positive review.

Edited June 16, 2008 by Leonard Kim (log)

[oakapple]

But, unless we can demonstrate that the "notionally anonymous" guys are turning out better and more accurate work than the non-anonymous guys, for me the argument in favor of presumed anonymity must fail.

←

Among regular reviewers in this town, Restaurant Girl and Andrea Strong are the ones making the case for non-anonymous reviewers, and I don't know anyone who thinks we need more reviews like theirs.

That's a bit of a cheap shot. I could easily say, "among regular reviewers in this town, Frank Bruni and Adam Platt are the ones making the case for non-anonymous reviewers, and I don't know anyone who thinks we need more reviews like theirs."

I guess I am struggling to clarify your position. You said:

I don't think it's necessary for a reviewer to make a reservation in his own name, nor to announce himself upon entry into the restaurant to give up the myth of anonymity.

This, in essence, is what Bruni and Platt do—as I understand it. If you don't think it's necessary that they they reserve in their own name or announce themselves, then what do they have to do, to "give up the [alleged] myth"? The next step is to post their mugs on their reviews, and then you're entering StrongBuzz/Restaurant Girl territory.

Whose reviews would you find more valuable:  Frank Bruni or David Rosengarten?  (Dave:  Rosengarten used to be the restaurant critic for "Gourmet.")

←

I don't think Frank's reviews would improve by shedding what is left of his anonymity.

[Fat Guy]

I don't think Frank's reviews would improve by shedding what is left of his anonymity.

←

I agree, but were it not for the Times's obsession with perpetuating the myth of anonymity we might have had a better critic in the first place.

[Sneakeater]

I agree, but were it not for the Times's obsession with perpetuating the myth of anonymity we might have had a better critic in the first place.

←

OK, now we're getting to the nub of this.

I'm all for attempted partial anonymity. I think it does encourage more representative experiences for reviewers.

But the problem, which hasn't been identified until now, is that the anonymity requirement eliminates many of the people who'd be most knowledgeable in the field.

The Times expects its main reviewer to sever all ties with the local restaurant industry. It's like it sends them off to a convent for the duration of their tenure. And, although I'm not privy to its criteria, if the Times is really serious about anonymity, it would have to reject anyone who's already well-known to the local industry, since unless they're willing to go around in disguise like Ruth Riechl they'd be instantly recognized without the need for surrepticious photos being hung in the kitchen.

Like FG, I think the Times would be able to hire more authoritative reviewers if it didn't care so much about anonymity.

How I ultimately come out on the issue I have no idea.

[Fat Guy]

More recently—although it's not quite what you're looking for—Frank Bruni did a quasi-scientific experiment at Le Cirque. He would send his friends in first, 15-20 minutes in advance, and see how they were treated—not well. Once the management realized they were with Bruni, everything changed.

←

Just to be clear, this is easy to do no matter what the critic's status. You can just send your friends on another night, or you can just talk to trusted acquaintances who've eaten at the restaurant.

By the way, the top restaurants recognize not only critics and other VIP types but also their frequent dining companions.

[Sneakeater]

I mean, imagine if the Times would hire as its theater critic only someone unknown in local theater circles. Or as its classical music critic only someone unknown to local musicians and presenters. The results would be laughable.

[Nathan]

I mean, imagine if the Times would hire as its theater critic only someone unknown in local theater circles.  Or as its classical music critic only someone unknown to local musicians and presenters.  The results would be laughable.

←

of course everyone at a given performance with reasonably similar seating gets the same experience. not so for restaurants.

[Sneakeater]

I was going to make that point myself.

Absolutely right.

It's a quandary.

[Fat Guy]

Sure but anonymity has become the tail that wags the dog of the enterprise of restaurant reviewing. At most, oakapple and co. have made a mild case for the occasional utility of anonymity as a tool in the reviewer's toolkit. But to allow that minor issue to dictate or even influence the choice of a critic is totally out of proportion to its best-case utility.

Of course I happen to think the oakapple case is wrong and that occasional random anonymity is on-balance destructive. But even if it's right it's such a minor benefit that it can't possibly justify the emphasis placed on it.

[Nathan]

here's what I hear:

1. that fuzzy pic of Bruni that shows up on Eater once in a while doesn't really look like him now.

2. with that said, high-end restaurants do usually spot him. they don't necessarily spot him right away. they don't spot him every visit (or they miss him for most of the meal on some visits)...this goes for even four-star places.

3. a busy, large restaurant, unless they have someone at the door fulltime with the sole job of looking for him (and who happens to either actually know what he looks like and has guidance such as that given at Nobu to their staff on how to spot his dining companions)...could easily miss him on some of his visits.

4. look at it this way, if he visits Per Se four times and manages to dine anonymously for just one of them...he now has quite a bit more data to work with.

Edited June 16, 2008 by Nathan (log)

[Fat Guy]

1.  that fuzzy pic of Bruni that shows up on Eater once in a while doesn't really look like him now.

There are lots of photos of Bruni in circulation, including a second one that Gawker likes to use as well as plenty of photos from restaurant and hotel security cameras. I remember when William Grimes became the critic and he was dining around the city (before he wrote any reviews) just to catch up because he knew so little about the New York dining scene, the people at Lespinasse figured out it was him and then went through the various lobby security videos at the hotel in order to get a full frontal shot, which was then circulated around to the fraternity of top restaurants. Before he had written a single review.

2.  with that said, high-end restaurants do usually spot him.  they don't necessarily spot him right away.  they don't spot him every visit (or they miss him for most of the meal on some visits)...this goes for even four-star places.

3.  a busy, large restaurant, unless they have someone at the door fulltime with the sole job of looking for him (and who happens to either actually know what he looks like and has guidance such as that given at Nobu to their staff on how to spot his dining companions)...could easily miss him on some of his visits.

Right. We know that once in awhile he's not recognized and that sometimes he's recognized late. And that he's recognized 80% of the time overall (or perhaps a little more or less).

4.  look at it this way, if he visits Per Se four times and manages to dine anonymously for just one of them...he now has quite a bit more data to work with.

←

"Quite a bit more data"? He has an arguably useless random sampling that, empirically, hasn't made his reviews any better than when he's recognized every time. "I wonder about Adour" hardly establishes anything to the contrary.

[oakapple]

Like FG, I think the Times would be able to hire more authoritative reviewers if it didn't care so much about anonymity.

←

If that is the reason why we have Frank Bruni and Adam Platt, then I would become a buyer of at least part of FG's argument: that is too steep a price to pay.

But there are plenty of extremely knowledgeable people following some version of the "Bruni rule," i.e., they don't adopt weird disguises, but they don't go out of their way to call attention to themselves. That still seems like the best system to me, for the reasons Nathan gave, a post or two upthread.

[Fat Guy]

But what you call "the Bruni rule" is not an accurate statement of New York Times procedure. The Times would never hire David Rosengarten, even though Rosengarten is a far superior critic to Bruni, because Rosengarten had a show on the Food Network and is widely recognized in New York restaurants. They wouldn't say "Well, it's okay because he'll just follow the Bruni rule."

[slkinsey]

4.  look at it this way, if he visits Per Se four times and manages to dine anonymously for just one of them...he now has quite a bit more data to work with.

"Quite a bit more data"? He has an arguably useless random sampling that, empirically, hasn't made his reviews any better than when he's recognized every time. "I wonder about Adour" hardly establishes anything to the contrary.

To be clear... What makes this a "useless random sampling" is that it's possible, for example, let's say that the critic is reviewing a restaurant with excellent service. But let's suppose that the restaurant's FOH staff has one of those rare "every so often" bad nights on the critic's first visit. Let's further say that they are back to their usual level of excellence for the critic's subsequent three visits, which also happen to be post-recognition visits. What is the quasi-anonymous critic to deduce from this? If he writes that the restaurant gives crappy service to non-VIPs, he is making an assumption based on his "useless random sampling" and writing something in his review that is not correct. There are any number of extremely plausible scenarios one could envision in which the quasi-anonymous critic's "quite a bit more data" based upon a (presumed) anonymous visit could lead to an erroneous review.

[oakapple]

4.  look at it this way, if he visits Per Se four times and manages to dine anonymously for just one of them...he now has quite a bit more data to work with.

"Quite a bit more data"? He has an arguably useless random sampling that, empirically, hasn't made his reviews any better than when he's recognized every time. "I wonder about Adour" hardly establishes anything to the contrary.

To be clear... What makes this a "useless random sampling" is that it's possible, for example, let's say that the critic is reviewing a restaurant with excellent service. But let's suppose that the restaurant's FOH staff has one of those rare "every so often" bad nights on the critic's first visit.

←

How is that sample any more useless than his sampling of the food? Suppose he pays one of his visits on the night that there's a substitute line cook manning the fish station, and all of the fish comes out tasting gummy? One bad night like that could be the difference between two stars and three.

[Fat Guy]

Everyone -- critics and readers included -- who has or learns of a bad service experience at a restaurant gives in to the human instinct to generalize from it. But an accurate statistical model doesn't.

If we assume that every restaurant -- even a four-star restaurant -- has incidents of bad service (for the purposes of a mathematical model let's divide service instances into "good" and "bad," though reality is probably closer to a point system, say 1 to 10) then every restaurant has what we could call a "service failure rate." Let's say a busy restaurant with 150 seats has 50 tables (average table size in a lot of restaurants works out to 3 and change, I believe) and does 3 sittings at dinner and 1 at lunch. That's 200 tables served per day. Assuming a 6-day week and some holiday closings we're looking at maybe 300 days of service. So that's 60,000 tables served per year, or 5,000 per month.

Just to make this simple, let's define our "period" as a month. Out of those 5,000 tables served in a month, how many are likely to experience service failure? At a restaurant with a superb service team, the number could be pretty low, maybe just a couple of tables a day. Having spent many a full service in restaurants that have won every good-service award, I can say with confidence that it's almost unheard of for a service to go by without some unfortunate disaster or another unfolding. But let's be exceedingly conservative and say the service failure rate for the period (one month) at our hypothetical three- or four-star-level restaurant with excellent service is 50.

Now let's say that at a two-star-level restaurant the service failure rate is 150 for the period. Triple the other restaurant's service failure rate, assuming same size and number of sittings. In other words we're defining -- just for the sake of this model -- three/four-star service as a service failure rate of 50/5,000 and two-star service as a service failure rate of 150/5,000. And the one-star rate, let's call that 500. Of course this oversimplifies, but it's useful for the illustration.

Okay, so as a visitor taking a random sampling, how many visits do I need to make in order to tell whether a restaurant has a three/four-star service failure rate, a two-star service failure rate, or a one-star service failure rate? I think we can all agree without resort to any high-level math that 1 instance of service failure has no bearing whatsoever on predicting whether we've just dined in a restaurant with a service failure rate of 50, 150 or 500. How about 2 or 3 instances? The one mathematician I asked said: "Unless the good restaurant failure rate predicts less failures than the number of critics visits, not sure how you could assess differences in the rates from that small number. If the good rate predicts less than 2 bad nights per period and you observe 2 or 3, maybe you can sort of say something. But if good rate predicts 35 bad nights and bad rate predicts 70 bad nights, and you observe 3 bad nights, I don't think you can statistically classify into good or bad." Feel free to dispute that, but it makes sense to me.

All the same modeling can be applied to certain aspects of food preparation. For example when critics write that such-and-such is "consistently" overcooked, they simply are not rendering a mathematically meaningful judgment as to consistency. Again, if there's an overcooking rate of X per Y at the best restaurant and Z per Y at a mediocre one, your visits at least need to approach X before you can distinguish between the best and the mediocre on the consistency point. No critic makes anything near that number of visits.

As I've said before, if you want to judge consistency, use a mechanism like Zagat but with a more serious approach. When you get 1,000 people responding, you get some meaningful numbers on the consistency point. Critics shouldn't be in the business of judging consistency -- it's mathematically not possible for them to do so -- so they should focus on higher-level issues that don't depend on a false concept of consistency. Which is exactly what other arts critics do.