The trouble with tasting panels

[Fat Guy]

Whether it's Cook's Illustrated, a newspaper or somewhere online, I rarely if ever agree with the findings of tasting panels.

On the one hand, I admire the conditions under which many tasting panels operate: blind, multiple tasters, etc.

On the other hand, a tasting panel is only going to reflect the taste of the group it represents. If you form a tasting panel from a pool of people who think McDonald's is the best restaurant in the world, the results will reflect those tastes.

How do you all feel about the tasting-panel results you've heard about.

[heidih]

Perhaps it sounds smug- but I rarely trust anything but my own taste. If the consortium or whatever the group is has chosen products in the past that I like I would give them some cred.

[Chris Amirault]

I feel like the bottom third is usually reliable: rancid, off-flavors, etc. seem pretty universal for most products. Middle to top it gets tricky.

Steven, have you ever been on a tasting panel? I've done a few for spirits and cocktails, and know for a fact that palate fatigue, order of tasting, and so on have a massive effect.

[Fat Guy]

I once did a tasting panel for a beer company that I can't talk much about because I promised not to. What I did notice was that there were a couple of people whose taste I trusted and we three agreed on pretty much everything, but we were a clear minority. The larger portion of people on the panel, who probably reflected popular taste, gravitated toward the Miller Lite end of the spectrum of preferences.

[David A. Goldfarb]

Reading the results of the butter tasting panel mentioned in the other thread, my first thought was "these people have really boring taste.

[Joe Blowe]

I agree with Chris: They're useful for avoiding the dreck, and for providing a list available products in a particular segment.

As with all things, YMMV, chacun à son goût, etc.

[jsmeeker]

They are certainly useful for getting a list of some products to try out as well as products to avoid.

Of course, it's not always perfect. As as been suggested, different panels can have a "bias", even if it's intentionally blatant. What's "best" is still personal opinion. People have ideas as to what a particular food ought to taste like. I recall a Cook's Illustrated taste test for chocolate. Scharffen Berger scored really low. It seems that the flavor was too "complex" for the panel. Speaking of chocolate and Cooks Illustrated... They have done a bunch of chocolate tastes tests over the year. As usual, they are frequently tied to some recipe they developed. It's interesting to see what appears to be inconistency from test to test.

[Dave the Cook]

The TV shows for Cook's Illustrated (America's Test Kitchen and Cook's Country) often feature the results of tasting panels, which are preceded by Christopher Kimball doing his own test of the trial products. I haven't done a serious accounting, but it seems like he disagrees with his own panel about half the time.