Does it make sense to rate recipes?

[Chris Hennes]

Over in the test forums we are discussing possible improvements to our recipe database, RecipeGullet. (Members can see one possible implementation here). One of the things we are discussing is a provision for members to rate recipes, but a point of contention is whether it makes any sense to rate recipes, considering the widely varying cultural standards. What do you all think? Does a rating make sense for a recipe? Would you pay attention to them? Would you rate a dish from a cuisine you weren't that familiar with? Is this a feature we should add?

[ElsieD]

Personally, I don't care for ratings unless they are accompanied by a comment. I have seen recipes rated on other sites where, say, a 2 out of 5 is given but without a comment it is meaningless. I would much rather not have a rating system at all but I would like to see comments.

[Tri2Cook]

I've never been a fan of recipe rating personally. I've seen way too many recipes where there's a low rating and then you start reading comments and most of the low ratings have things like "I didn't have x so I used y" or "it said to do this but I decided to do that instead" or similar things. Despite indicating they didn't actually follow the recipe, they still gave it a low rating. I added my own review to the boiled water section as an example of what I mean. But I'm not actually against a ratings system, just not likely to use it.

[gfweb]

I'm against a rating system for all the reasons stated above. In addition, it isn't eG-ish, its Yelp-y.

 

Comments would be helpful.   ***/***** wouldn't.

[Lisa Shock]

Personally, I don't care for ratings unless they are accompanied by a comment. I have seen recipes rated on other sites where, say, a 2 out of 5 is given but without a comment it is meaningless. I would much rather not have a rating system at all but I would like to see comments.

 

The software allows for and asks for a review along with a star rating. Under each recipe is also a tab for comments without stars. I see the comments as being useful both ways. Without stars the comments allow someone to ask about substitutions, or how well the dish freezes.

We got into a discussion about some guidelines I wrote for the star system. I have found guidelines for such systems to be useful in other places.

 

Here is what I wrote, I meant it as a framework, a place to start. This wasn't a fully finished masterpiece or anything. And, it has drawn comments.

 

5 Stars

The recipe has clear instructions for consistently reproducible results. The food made by following the recipe is of the highest quality overall: in taste, aroma, texture, and appearance. The world’s finest restaurants would gladly serve this item.

4 Stars

The recipe has clear instructions for consistently reproducible results. The food made by following the recipe is generally of exceptional quality. Most restaurants would be proud to serve this item.

3 Stars

The recipe has clear instructions for consistently reproducible results. The food made by following this recipe is of good quality. Many restaurants may serve this type of item, it would be considered respectable while also being fairly basic.

2 Stars

The recipe may have minor consistency or clarity issues. The food made by following this recipe is edible but unremarkable, or inherently flawed in some way. A restaurant might serve this food, but customers are not flocking to buy it.

1 Star

The recipe may have several issues. The food made by following this recipe is substandard by most common measures, and would not be featured in a cookbook for home use nor served in a restaurant.

[gfweb]

Those would be helpful stars if people would stick to them faithfully but I have doubts that they would.

 

There's also the problem that most recipes if judged realistically, even from this distinguished group, would be two-stars or maybe 3 at most. But friendly charity would creep in and grades would inflate.

 

I still am in the no stars camp.

[Lisa Shock]

The conversation started with me commenting that I'd like a way to indicate that some of the recipes (notably by people who post 2-3 messages and are never seen again) are really terrible, from any cultural perspective. Then some that won't give consistent results due to vague measurements or instructions, like "add two handfuls of pasta."

 

The opposition poses the question of who is anyone to be rating a recipe, particularly one from a culture of which one is not a native? And, how does anyone know what would be served in the best restaurants, again outside of one's cultural circles. 

Edited November 16, 2015 by Lisa Shock (log)

[Tri2Cook]

Like I said, I'm not opposed to it, just very unlikely to ever use it. It may be helpful to some. If so, it's probably worth doing. I've just never encountered a recipe site where the ratings were a particularly accurate representation of recipe quality.

[liuzhou]

Agree with the negatives. Very Yelp-y indeed.

[gfweb]

The conversation started with me commenting that I'd like a way to indicate that some of the recipes (notably by people who post 2-3 messages and are never seen again) are really terrible, from any cultural perspective. Then some that won't give consistent results due to vague measurements or instructions, like "add two handfuls of pasta."

 

The opposition poses the question of who is anyone to be rating a recipe, particularly one from a culture of which one is not a native? And, how does anyone know what would be served in the best restaurants, again outside of one's cultural circles. 

 

My opposition is not over who are any of us to rate a recipe. I actually think we are a great group to do just that. Cultural differences are an issue of course.

 

My problems stem from long experience in validating subjective grading systems. Its hard, really hard, to get a group of people to apply a uniform subjective grading system to anything...even if you have a meeting to harmonize individual variations and talk about it a lot. Even then the results are often crappy.

 

Consider  grading black vs white.  If there are only two choices, most will say anything not pretty black is more white than not. If there are three options, black, white and grey...it gets easier..there is pitch black and pure white and then everything in between. But if there are 4 or more options it gets murkier again. And you have to ask what have you accomplished.

 

So, still I like written reviews.

[nickrey]

Sometimes recipes are badly written, leave ingredients out, etc.

 

However, in many cases success or otherwise of a recipe depends on the skill of the cook and the quality of ingredients used.

 

I can't see the benefit of someone rating a recipe based on what they did when they were wielding it.

[liuzhou]

Surely the ability to 'like' any post is enough. If a lot of people 'like' a recipe I may have a closer look (providing the name of the dish doesn't indicate that it's something I really dislike). If no one likes perhaps I'd skip over it (unless the name of the dish indicates one of my favourites).

1 to 5 rankings are pretty meaningless. Comments are much more useful.

[Lisa Shock]

Right now, comments are disabled in Recipe Gullet.

[Lisa Shock]

I should point out that a rating system was not my suggestion. My original suggestion was to allow recipes to be moved to a newly created section, maybe called 'curated recipes', if a certain number (maybe 3) members tested and confirmed a recipe. This would hopefully elevate the good stuff out of the current slushpile which has some subpar content.

 

Chris pointed out that we don't have staff to curate such a section and suggested the star system and comments, both of which are parts of the new software which we could use. (samples on the test site)

[Thanks for the Crepes]

I definitely like the ability to comment, and I find ratings of little value without discussion of why the rater placed that value on his/her rating.

 

Comments alone would add a lot a value to me, since I could ask questions of the author before investing time and money in the ingredients.

 

This group is a skilled and astute bunch of cooks, and when they put their heads together, can come up with very impressive culinary creations. I didn't like roasted cauliflower very much after reading the original rendition of the recipe, but after perusing the entire very long thread, with everyone's tips, tricks and refinements, it's one of my favorites.

 

Also, remember, this is NOT Yelp. We know each other better, and a recommendation from one member may carry more weight than another based on our individual preferences and knowledge of the author's food philosophy and known practices. Even on Yelp, I have certain reviewers I know and look for their posts. I live in a much smaller community than, say, NYC where that strategy probably wouldn't work at all, but eG is an even smaller community than mine.

 

It's fine with me if the ratings go away since so many members I like and respect seem to want them to. I understand it could be very hurtful to have a recipe you spent time and effort generously offering up rated badly. I agree that ratings do seem a little un-eG. When I first discovered the Cook-Offs, I was delighted that they were a think tank instead of a competition, which the topic title called immediately to mind.

 

I am lobbying to keep the comments, because I think they add tremendous value.

 

Edit: I believe most recipes are currently posted in the threads, and not in Recipe Gullet. Perhaps the ability to ask questions and discuss there would encourage more actual use of it and allow us to find them more easily?

Edited November 16, 2015 by Thanks for the Crepes (log)

[Smithy]

<snip>

I am lobbying to keep the comments, because I think they add tremendous value.

 

Edit: I believe most recipes are currently posted in the threads, and not in Recipe Gullet. Perhaps the ability to ask questions and discuss there would encourage more actual use of it and allow us to find them more easily?

 

The advantage to RecipeGullet is its searchability - by recipe title, tag, topic or author.  It's harder to find a recipe when it's buried in the Dinner topic or some other topic that doesn't pertain to a specific food.  On the other hand, recipes posted in those topics allow discussion and questions.  I'm with you: I'd like to see comments allowed on RecipeGullet. The ratings are of less value to me - but, as noted in earlier posts, this crew is astute enough that I'd place more value on ratings as long as they included reviews and member names.

[Mjx]

I can see ratings being useful only if they reflect objective aspects, such as reliability in terms of delivering what was described as the outcome, and even then, you have assume a certain amount of uncertainty (individual error, inconsistencies owing to use of volume measurements or unreliable equipment). It's so easy to get caught up in subjective reactions (e.g. I've found ATK's recipes consistently reliable, at least when they also include weight measurements, but I haven't always loved the results, sometimes finding savoury things too sweet for my taste).

[gulfporter]

A friend of mine worked as a reporter for a small mid-Atlantic newspaper (city had 200K population).  Once a week, each reporter had to take incoming phone calls from readers.  He said by far the largest volume of calls were about errors in the paper's recipe column where reader-submitted recipes were reprinted.  A cup of flour when it was supposed to be sugar; wrong temperatures; missing ingredients; not enough instructions.  People were angry that they spent their money on ingredients for a product that was inedible.  I took my friend's story to heart when I started writing a recipe column for an expat monthly magazine when I lived in Mexico; my DH and I read and re-read my submissions before finalizing.  In five years, one boo-boo got through (a missing ingredient that was referred to in the text, but left off the list of ingredients).  

 

I like to think I can spot a botched recipe, but typos happen and I think it's good to allow comments for recipes that are unclear or found to be erroneous.  I'm not in favor of ratings for the reasons others have stated.  

[KennethT]

As others have stated probably better than I can, I also would like comments but not ratings. Even with a suggested framework for ratings, I think it is difficult for most people to objectively look at a recipe for something that may not be to their taste (too sweet, too spicy, too salty), but may be consistent with the style of cooking or region of origin, and rate it highly.

It is hard to wrap one's mind around the concept of rating something against what it is supposed to be, rather than your personal feelings about it. This gets even more complicated when the recipe is for something that you're not intimately familiar with - like something you've only had in a local restaurant once. Then your comparison is only to the 1 example, not to a giant sample of examples.

This is even more complicated by availability and consistency of ingredients. Take chilis, for example. Of the same variety, some are much hotter than others. This makes it very hard to standardize a recipe - such as 3 grams thai bird chilis, chopped finely. That's why a lot of cuisines rely more on taste than recipes - this is a hard concept for a lot of people who are used to dealing with ingredients with little variation. I think that most of us here on eG are good enough cooks to know that you need to keep tasting to make sure a dish is heading in the direction you want, and not to follow a recipe blindly - but the problem is that the more specific a recipe is, the more the reader does not want to deviate from it - even though, in its specificity it is highly repeatable assuming consistent quality of ingredients.

[lindag]

I like ratings with reviews.

[Martin Fisher]

Comments but NO ratings!
As others have said, ratings are totally subjective and folks often don't follow recipes as written, use ingredients of different quality, use different volume measuring techniques (such as scooping rather than spooning and sweeping flour), etc. etc. etc.

 

Folks changing a recipe in several ways (rather than rating it as written) and then complaining loudly about how bad it is and how much they hate it is one of my biggest pet-peeves!!!!

 

[Deryn]

My vote is for comments, no ratings - for all the reasons stated by everyone above, as well or better than I could.

[blue_dolphin]

Ditto.  

I'd love to see comments utilized for the recipes.  For me, numerical ratings would not add value but I'd have no problem just ignoring them if they were of value for others.

[cakewalk]

I much prefer comments, but I can't say I discount ratings entirely. The ratings (and comments) for the eGullet recipes would come from membership only? If so, I think that makes it very different from what we find on most other forums. If I'm reading through ratings and comments for recipes, I'm always more interested in the lowest ratings. I want to know what the problem is. If it's silly (they substituted ingrendients or methods on a whim), I can just ignore it. But often it's a thoughtful critique, and it's the low rating that called my attention to it in the first place. It's not necessarily informative to go through pages of five-star ratings where everyone says, "this is the best recipe I've ever made in my life!!!!"  

[MelissaH]

Comments yes, ratings no. For all the reasons already stated, a rating without some reasons behind it is a meaningless number. That said, those who like to give ratings could certainly include one within their comments.