11 replies to this topic

[IndyRob]

I'm surprised I couldn't find this topic already. Sorry if I missed it.

So, I'm sitting here trying to decide between two restaurants. They're both steakhouses but one is a little more upscale and expensive, while the other one is less so, but seems well respected.

I find them both on Bing Local, look at the menus and notice the ratings. It turns out that both have about 30 reviews and on the food quality scale, the more expensive option scores just over 2 1/2 stars while the less expensive one is nearly a solid 5 stars.

The thing is, we have a personal recommendation for the more expensive option by someone who is well traveled and certainly has much fine dining experience. Oh, and his son is a chef.

So I have a little quandry here. Do I go with the one personal recommendation, or go with the numbers?

What are your experiences with web ratings?

[nibor]

People go to expensive restaurants, then go online and complain about the bill. These are the same people who complain about portion size.

[ChrisTaylor]

Well, yes. You need to consider who is scoring the restaurant. Cheap and cheerful is what many people like. Urbanspoon is great in that it conbines a crowd-sourced score with amateur and professional reviews. You can look at a review to see what makes people give a restaurant a low score--maybe it's the bill.

[IndyRob]

Well, as it turns out, after my wife got home I found out I had it backwards. The one recommended was in fact the higher rated one.

My bad, but it does provide an anecdotal correlation. In this case, the reviews may have steered me to the correct choice.

Having two sources that differed wouldn't have said as much (without visiting both restaurants).

[MaxH]

I think IndyRob's comment and topic gets at the very core issue in consumer restaurant guidance. Here are some observations, maybe familiar to many other restaurant customers, who have also struggled with these information sources. "Aggregated" restaurant rating sites, like the one IndyRob checked, arrived on the Internet soon after HTTP tools and "browsers." A very serious one operated here in the SF Bay area (and a few other metro regions with large online populations) in the middle 1990s, a little early for the general public, but it showed exactly the same issues. (FYI in the 15 years before that, as Internet access slowly broadened from basically technical academia into mainstream, its typical communication tools were not "sites" but variations of email, like newsgroups [broadcast email]; those tools carried plenty of restaurant commentary, but as individual anecdotes, not aggregated into numerical "ratings.")

Formerly, we consumers turned to centralized experienced sources for restaurant guidance -- Guide Michelin etc., Mimi Sheraton, in my region sources like Jack Shelton's newsletter and Ron Riera (a talk-radio critic who was a restaurateur himself and also knew the regions' restaurants encyclopedically). Whatever their biases, you could get to know these sources and how to correlate their advice with your own experiences.

Enter Zagat guides (in print), then today's aggregating Web sites. With these sources I've found you're basically comparing apples to bicycles to annuities to sonatas. The source consistency of the experts is missing. If even many of IndyRob's 30 reviews per restaurant had actually tried both, so as to be competent to compare, that would be unusual in my experience. You get a different self-selected set of commentators for every restaurant, the conscientious ones drowned out by the offhand, or those with weird chips on their shoulders, or who made trouble themselves at the restaurant and want revenge, or employees or ex-employees shilling or slamming businesses where they have personal interest. Near me, in a cluster of radically different Asian restaurants heavily commented on Yelp (typically at least 500 reviews), the averaged "numerical" ratings are meaningless, all converging to 3.5 stars, a useful statistic about the raters' habits, not the restaurants.

The solution? I've found real value in sites like that by identifying conscientious or reliable individuals even though they're invisible in the averages, and following their restaurant comments specifically. It seems like the knowledgeable individual critic's value still shows, even in vox-populi sources.

[Edward J]

Max H as an excellent post and touched on many nuances, but forgot item: Advertising.

Websites are a business and need to earn some kind of money. Content is usually free, provided by posters and bloggers, but you have to hit someone up for money. Can you guess who?

Now with one particular site--not naming any names here, but it sounds like something that comes out of a small dog's mouth---my personal experience (as a business owner)was less than stellar. When we first opened up we got rave reviews--4's and 5's. This earned us a personal e-mail from the site asking us to advertise with them. We declined, the offer was put up again, politely declined again, and then many of the positive reviews were deleted. I've had several customer complain to me that they put up a great review on our place and then a day later the whole thing was removed. Then the negative posts come in, some really personal attacks suggesting divorce and/or suicide, and yet the reason for this is never explained. I understand the same site in many other cities does not operate this way.

I also make a very clear definition between a pro-blogger and a hobby blogger--I much prefer the hobby blogger. Why? They have a regular full time job, that's why. The pro blogger wants compensation for his/her work. Can you guess what that would be? Best way to deal with the pro-blogger's requests is to have a request of my own: A disclaimer on their blog saying that the blog was partially or fully "sponsored" by my business. Works like copper on snails....

[SJMitch]

Review sites like Yelp can be helpful, but you have to put in a lot of time or do intelligent searches to get decent info out of them. I don't find much value in the overall ratings, but do get value out of some of the detail in the reviews.

Oddly, I often find good restaurants to try based on bad reviews on Yelp. Such as reviews of Ethiopian restaurants that complain their injera is 'too sour' (what? they make their own and actually ferment it properly instead of dousing it with soda -- I'm there!). Or 'bad' reviews of Pakistani dishes or Mexican goat stews complaining about bones, or complaints about inedible parts in soups that should have them (e.g. lemongrass, thick ginger slices) -- instead of warning me away it tells me there is likely to be well-spiced and flavorful dishes to be had. One review of a local coffee place complained that they were weighing the beans before making a pourover and about the time it took -- the reviewer thought they should make a batch in advance and should just know how much to put in. From that bad review, I knew that was a coffee shop that really cared about the quality of the coffee they served. I also find complaints about 'small portions' to often be an indicator of quality as long as it isn't a sudden change for the place (it's not a perfect signal, but it's once piece of evidence). Also, I've had better luck following the recommendations of reviewers that are more balanced and are able to include some of the shortcomings of the place in the review (no place is perfect). Then I can decide how important that fault is to me. Restaurant completely family run and has disinterested kids as servers and can't handle lunch rush but parents can cook up a storm? -- only go off hours.

Before I learned to read the reviews critically, I was in a new town and tried a Mexican restaurant with an overall good rating. The food was sad. I wrote my first Yelp review, being detailed about what was wrong with the food and giving them a low rating. It was promptly 'removed', or in Yelp terminology, 'filtered'. I lost interest in Yelp for a few years until I learned that on Yelp, reviews from new users often don't stick until the user establishes a track record of many reviews and had other indicators they are not a shill just for or against a specific place (it helps to have a photo on your profile, use a non-joke handle and fill out your profile more). To test it, I started to review to other places and cleaned up and filled out my profile. After 10 or so reviews and regular use of the site, my 'bad' first review from a few years ago was reposted. You can't expect to join Yelp write a single review strongly pro or con and have it stay long term. They have automated filters against new users to keep shills out and it catches most new users.

There is an appliance store near me that solicits reviews. They have over 100 'filtered' and unseen reviews. I guess they are hoping they get enough on an ongoing basis that the day or two they stay up will be enough to sway readers. Soliciting reviews is against Yelp policy. When I see 100s of 'filtered' reviews, I know it is a place to avoid. Really good or really bad places will always have a few 'filtered' reviews since those places entice new reviewers in who don't know that just reviewing 1 or 2 places with a blank profile won't work.

I like now having an established track record on Yelp and knowing that if a place knocks my socks off and I write a total fanboy review it will stick. (same for bad ones).

One thing I don't like about Yelp is that I now can't be completely honest -- if a small local place is decent but not steller and yet I'm likely to frequent them, I either can't rate them or have to rate them highly since they will know it is me and for these places it would be too rude to do otherwise and generate bad feelings (the new places now always seem to follow Yelp with a bit too much interest). I usually rate them 5 stars, start off with all the good stuff and then try to have a vague comment or two about their shortcomings to try to be a bit more helpful to readers. Restuarants have offered me perks for my reviews -- thankfully all of them have been after the review was written. It always makes me uncomfortable since I can't always tell if it is for the review (which I will then decline) or because I am seen as a regular (which I will accept).

[Edward J]

I have difficulty in comprehending the above rules and regulations.

I do understand about shill-ing new places and understand that they are removed, but I don't understand why negative reviews can't be removed as in my case they were really personal attacks and had no information to back up vague claims of "rudeness". If I put arsenic in your coffee, fair enough, it is rude, but asking someone politely to wait until I fill out the first two guest's orders is not rude, but only professional and polite for all those involved.

Who has their finger on the "delete" button, and is it in any way connected to advertising with the featured business?

[MaxH]

One thing I don't like about Yelp is that I now can't be completely honest -- if a small local place is decent but not steller and yet I'm likely to frequent them, I either can't rate them or have to rate them highly since they will know it is me and for these places it would be too rude to do otherwise and generate bad feelings (the new places now always seem to follow Yelp with a bit too much interest). ... Restuarants have offered me perks for my reviews -- thankfully all of them have been after the review was written. ...

Welcome to the realities of being an attentive restaurant diner. I'm coming up on 20 years now of restaurants noticing that I was paying close attention, or was introduced by someone they knew such as a local journalist, then second-guessing that I must be a professional shopper, and offering payola, unsolicited. (Once at a high-end restaurant in Europe where I was simply learning about the cuisine, this even led to a stand-off, resolved eventually with all dignities intact, by a compromise bill.)

I find this annoying, as though assuming my good opinion is for sale for a few dollars or a free dessert, which has led to routinely refusing even perhaps innocent comps if there is any possibility of my writing public or private recommendations, which these days is usually. I just gently refused such an offer at a local place I frequent, for that reason. More annoying is when some bloggers stumble into this reality and try to milk it for comps -- at least it clarifies just what their opinions are worth.

The Y-site actively promotes contributor pictures, including that among criteria for specially featured reviews and premium reviewer status, whatever that means. Regulars work around the rule by changing their pictures later or using illegible ones, but as SJMitch observed, reviewer photos conflict fundamentally with objective criticism. Maybe Y'ers prefer being recognized (like those bloggers I mentioned), but the policy does reveal something about the site's prioritiies.

[SJMitch]

Although I like Yelp, I consider 90 to 95% of the reviews on Yelp to be nearly worthless. For folks who are willing to put in the time to find the signal in the remaining 5 to 10%, Yelp can be helpful (I mostly use it to find new places, hole-in-the-wall shops that serve unique cuisine not broadly available, unusual dishes or preparations, shops that use ingredients from other local suppliers I admire, or to fulfill cravings for specific ingredients -- e.g. "I want some a-choy today, can I find a Taiwanese place with it on the menu?"). For many people, it's just not worth it. Yelp's power is overrated.

It's really not worth it for a business to stress out or even waste any effort or concern about a bad or misleading review. This is especially true when the opinion expressed is subjective and vague. Savvy readers weed these out on their own. It's similar to reviews on Amazon. You can tell when someone just doesn't get it about a product on Amazon or has an ax to grind. If you get a bad review, even if it is unfair, let it go. Trust that the type of customer that you'd want at your business is smart enough to discriminate between reliable and unreliable reviews.

The only time it might hurt business is when it is a trend among the majority of reviews. This typically happens when there really is an issue with the business or when someone has figured out how to game the system and is willing to open lots of a accounts and even spend 10s or 100s of hours to build the reputation of those accounts, all with the intention of attacking a competitor. But then it's not just one or two reviews. It's dozens or hundreds, with new ones added all the time. Small fry shills are usually caught in the same filter that prevents passionate but genuine new Yelp users from getting their first few reviews posted.

[gfweb]

I find chowhound helpful. There's a lot of promotional crap that sneaks on but I find it easy to filter that out.

[MaxH]

SJMitch, I gather that you, like me, take time to study Yelp comments and try to sort wheat from chaff. I also don't know what fraction of busy readers actually do that, desirable though it may indeed make them as customers. I find it challenging to sort out quality comments from the hundreds posted per restaurant when I'm exploring new territory -- how many people actually do that? And I also notice the type of petty or sleazy attacks that Edward J mentioned, pure smear with no concrete information; they impair the site's utility to me as a customer too.

I too notice that "the new [restaurants] now always seem to follow Yelp with a bit too much interest." I've heard from owners anxious over their numerical ratings, claiming that customers cite Yelp for the referral, or that the numerical ratings affect their business. Owner complaints like Edward's about pushy advertising sales have been commonplace. And "gaming" takes many forms: I've noticed someone, I don't know who, waging a protracted oblique smear campaign against one restaurant -- it could be anyone. There's specific history locally both of employees instructed to smear competitors via Yelp, and of both problem employees and problem customers waging Yelp vendettas. This particular gaming is unmistakeable, and clever enough to point to no specific source. I may discuss it with the restaurant.