Expensive Wines Wasted on Untrained Tasters?

[slkinsey]

I happened upon an interesting working paper from the American Association of Wine Economists today entitled Do More Expensive Wines Taste Better? : Evidence from a Large Sample of Blind Tastings.

Here is the abstract:

Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less. For individuals with wine training, however, we find indications of a positive relationship between price and enjoyment. Our results are robust to the inclusion of individual fixed effects, and are not driven by outliers: when omitting the top and bottom deciles of the price distribution, our qualitative results are strengthened, and the statistical significance is improved further. Our results indicate that both the prices of wines and wine recommendations by experts may be poor guides for non-expert wine consumers.

[Florida Jim]

Our results indicate that both the prices of wines and wine recommendations by experts may be poor guides for non-expert wine consumers.

If you ask me, they are fairly poor guides for more experienced wine consumers, as well, regardless of the theory espoused in the abstract.

For myself, I can say that it has been at least eight years since I have read any of the professional reviewers in print and that my price point per bottle of wine in my cellar has moved lower over that same period, despite euro/dollar conversion rates and the evident increase in wine pricing overall.

Best, Jim

[mkayahara]

Is this related to Robin Goldstein's book The Wine Trials, or did it simply draw a similar conclusion? Eric Asimov had an interesting discussion of the book on his blog, here and here.

Edit: Or I could have just re-read the blog posts in question and realized that they are, indeed, related...

Edited July 16, 2008 by mkayahara (log)

[Fat Guy]

I totally believe it. Fine wine is an acquired taste. It's not as sweet, not as direct, etc., as cheap wine. That's why young people start out with wine coolers and such. I'm not surprised that the average person would prefer a $10 Australian Shiraz to a Rhone costing five times as much.

[slkinsey]

It also seems true, to my admittedly less-than-expert taste, that price is often a very poor indicator of quality.

[docsconz]

It also seems true, to my admittedly less-than-expert taste, that price is often a very poor indicator of quality.

←

Interesting statement, Sam. Since you used the word "often" and not "usually", I will agree with you. I think that this is more often the case where wines have a greater hype than others. For example, while I enjoy many wines from the Napa Valley, it seems that a certain caché has developed from being a winemaker and having expensive wines - the more expensive the better. With an approach that if the wine is cheap, it can't be considered to be good, many wines that perhaps should be cheap aren't. I think this holds less true in areas with longer winemaking traditions, that are less trendy such as Alsace, the Rhone or Germany. In areas like that, the price:quality ratio is more likely to be accurate. One is more likely to find that the increased price is reflected in increased quality per the standards of the area. Whether or not the differences in quality are worth the difference in prices is another question altogether.

[Fat Guy]

My general take on that aspect of things is that, when you lump all the world's wines together, cost is not such a great indicator of quality. Exchange rates, differences in productivity, etc., ensure that. But I also think, in general, that if you narrow the focus to, say, wines from a given region and grape, quality and price do tend to stratify somewhat reliably. There are sometimes outliers, perhaps because some risk-taker picked early or whatever and made good wine in a bad year but doesn't have enough of a reputation to command a high price. But those are the exceptions. You just don't find a lot of $25 California Cabernets that outclass the $75 ones by the standards of even moderately experienced tasters. Still, plenty of people will prefer the $25 one, and the $12 blend to that. That, I think, is the take-away from this study.

[Craig Camp]

I think wine is no different than food. Someone who has never tasted fine cuisine will probably be taken back by El Bulli. The same goes for wine I have seen many inexperienced tasters prefer simpler wines to more complex ones simply because the more complex wine just has too much going on. If you are accustomed to food and wine as background music you can be shocked when in comes to the forefront and you may not like it.

To second Jim's point, price often has little to do with complexity these days.

[artisanbaker]

I can't say that I agree with every point made, but what sticks out to me is that my wife's tastes indicate the more common consumer.

As it is common knowledge that my wife is always right, it is challenging to harness her preferences so that we have some common ground. This common ground is important so that I do not purchase wines that only I like; I need to open wines that provide pleasure to her as well.

Well, she can get hung up on labels and doesn't like tasting blind as she is "worried" about being "wrong," even if it only me that is around.

She doesn't follow the world of wine like I do, and she usually doesn't necessarily make a qualitative remark when I open a Giacosa Barolo Faletto or a Dugat Lavaux St. Jacques.

So in summary, she has label association that is hard to break and this qualitative association supercedes what's in the glass.

"For better or for worse..."

[Florida Jim]

Well, she can get hung up on labels and doesn't like tasting blind as she is "worried" about being "wrong," even if it only me that is around.

This was the case with Diane for years. I suspect that what finally got her over the hump was to hear me criticize wines that were very highly touted or very expensive or both.

Well, that and developing her own palate experience, which is the key to it all.

Best, Jim

[artisanbaker]

Thanks Jim.

I am optimistic that she will come around and when she does choose to disclose her opinion on in a blind tasting of an expensive bottle, I value her input. Probably a matter of confidence at some point.

Best to you, too.

rob

[Brad Ballinger]

For myself, I can say that it has been at least eight years since I have read any of the professional reviewers in print and that my price point per bottle of wine in my cellar has moved lower over that same period, despite euro/dollar conversion rates and the evident increase in wine pricing overall.

Best, Jim

←

Ditto for me. And I end up going out to dinners with some Twin Citites wine enthusiast types where we each bring wines. I feel like a cheapskate at times since the wines I'm bringing are generally lower-priced than the industry or media darling wines brought by others. Yet, I've often found the wines I bring "hold their own" or are sometimes preferred to wines that costs two or three times as much.

[KatieLoeb]

Helping to choose wines for a small wine bar that has 40 bottles or so by-the-glass at any given time has absolutely reinforced my opinion that price and quality are not necessarily directly related. Certainly for my purposes professionally, I have to find wines that are at a price point that allows the restaurant to serve them by the glass, and that will appeal to the most people - both the novice wine drinker that comes to a wine bar to actively learn and the more experienced drinker that comes there to try interesting wines they might not have seen before or to see which examples of their favorite varietal/style of wine we might have that week. I'm further restricted by the fact that our list is all French, Spanish and Italian. But there's plenty of interesting wine from various regions in each of those countries in the $8-$20 price range that hold their own just fine with both the discerning and the less educated consumer. Reasonably priced Super Tuscans, interesting Rhone blends, "just-outside-the-lines" French DOC wines and Spanish Tempranillos/blends are my friend. And possess an excellent QPR for both the restaurant and the guest.

[JohnL]

I don't know any wine reviewers who do not review inexpensive wines regularly.

In fact many of the wines that "won" in these tests have been favorably reviewed by the critics (especially the Wine Spectator).

So there is ample evidence that one would do well (at least as well) following the recommendations of the Wine Spectator.

also note the conclusion:

"...individuals on average enjoy more expensive wines SLIGHTLY LESS.

Among folks with a little wine knowledge "....we find evidence of a POSITIVE relationship between price and enjoyment."

[slkinsey]

Um... I think it's clear from the article that they are not talking about "folks with a little wine knowledge." I have a "little wine knowledge." Their "experts" were those who had taken "some form of [formal] wine training, such as a sommelier course." This is more than "a little wine knowledge."

More to the point, the authors also make it clear that tasters of all stripes routinely enjoy more and give higher ratings to more expensive wines, when they are aware of the price (even when it is the same wine). One shouldn't discount the possibility that their "expert" tasters had been trained to detect the properties that are most associated with more expensive wines. Otherwise, the price/rating correlation might not be so strong, in consideration of the fact that we all acknowledge that there are some less expensive wines that are better than higher priced ones.

Indeed, the authors conclude by asking: "These findings raise an interesting question: is the difference between the ratings of experts and non-experts due to an acquired taste? Or is it due to an innate ability, which is correlated with self-selection into wine training?"

Edited July 28, 2008 by slkinsey (log)

[Holly Moore]

I am untrained in the ways of fine wine - one of those boorish "I know what I like" gulpers.

Having amassed a bunch of years and, within those years, many fine dining opportunities, I have had the good fortune to sip many great wines - expensive wines that are supposed to taste very good. They did. I can not explain the subtleties of why? I can not legitimately fill out a grading sheet. I just know that I liked them.

I seem to enjoy fine, expensive wines. A lot. Ergo they were not wasted on little ol' untrained me.

[Bill Miller]

I feel very lucky, I've enjoyed both Caymus Cabs-(in multiple years), Cavollotto Barolos (12 years old--I still have 2 I'm saving), Domaine Drouhan Pinots,and Silver Oak Cabs. But Gabbiano Chianti and Firesteed Pinot still taste good to me at the right time. There is joy in simplicity to the learning tongue as well as complexity. By the way I've never spent more than $175 for a bottle-the Gabbiano is $8 at Specs and great with my home wood burning pizza oven baked pizza.

Edited July 29, 2008 by Bill Miller (log)

[JohnL]

Um... I think it's clear from the article that they are not talking about "folks with a little wine knowledge."  I have a "little wine knowledge."  Their "experts" were those who had taken "some form of [formal] wine training, such as a sommelier course."  This is more than "a little wine knowledge."

More to the point, the authors also make it clear that tasters of all stripes routinely enjoy more and give higher ratings to more expensive wines, when they are aware of the price (even when it is the same wine).  One shouldn't discount the possibility that their "expert" tasters had been trained to detect the properties that are most associated with more expensive wines.  Otherwise, the price/rating correlation might not be so strong, in consideration of the fact that we all acknowledge that there are some less expensive wines that are better than higher priced ones.

Indeed, the authors conclude by asking: "These findings raise an interesting question: is the difference between the ratings of experts and non-experts due to an acquired taste? Or is it due to an innate ability, which is correlated with self-selection into wine training?"

←

I don't think much of anything in the article is "clear."

Asimov does a good job in critiquing the whole mess.

Their definition of "expert" is "a participant who has had SOME FORM OF WINE TRAINING..."

In fact their universe of participants contained professionals (academics, lawyers, doctors etc), travel writers, food and wine bloggers etc. Many of these folks are highly likely to have considerable experience with wine--far more than novice or average wine drinkers/consumers.

They concocted a method to determine each participants tasting skills involving Yellowtail which acts to further muddy the waters and in light of a better participant selection process would be unnecessary.

The "winners?"

Fat Bastard Chardonnay, Charles Shaw cabernet AND the Chardonnay, Bogle Zin, Almaden Chardonnay, Fetzer merlot, Mondavi, Louis Latour, etc etc etc.

What wines did they win out against?--who knows?

What did the group think about the more expensive wines--who knows?

By what margin did each inexpensive wine win by--????

One huge problem with the conclusion is that the economists and Mr Goldstein completely mis read the straw men they set up--the professional wine critics and the magazines. The fact is, as I noted earlier, most of the wines that "won" these 'trials" were reviewed favorably by the critics and are, in fact, often highly recommended. Goldstein et al don't seem to realize that 85 points indicates a "very very good wine many of which are great values..." (Parker). This indicates that the expert reviewers and the participants in the trials are basically in agreement. Thus one main conclusion is reached by Goldstein et al is WRONG.

The one "prestige" wine singled out is the 1999 Dom Perignon (a Veuve Cliquot is also noted). This is the wine that Goldstein selects to make an example of. It is the ONLY more expensive wine with tasting notes included in the book. Under "nose" is this line explaining its performance:

"If you use your imagination, it's not hard to see how those overwhelming aromas of cream and toast can turn ugly for some."

I think this reveals quite a bit about Goldstein and his cohorts at Princeton and especially about the "wine trials.".

[Darren72]

If you ask me, they are fairly poor guides for more experienced wine consumers, as well, regardless of the theory espoused in the abstract.

I think you mean to say "regardless of the data espoused in the abstract."

[JohnL]

If you ask me, they are fairly poor guides for more experienced wine consumers, as well, regardless of the theory espoused in the abstract.

I think you mean to say "regardless of the data espoused in the abstract."

←

Ironically, I don't think the paper shows this at all. The fact is most every professional critic I know of reviews inexpensive wines and recommends them all the time. In fact, most of them are quite enthusiastic about pointing out good values and bargains worth trying. There is little evidence to support the notion that any critic is a "poor guide" especially if one accepts that informed opinion from any quarter is basically a suggestion to try/sample a particular wine.

The real problem with this "abstract" is that its raison d'etre is rooted in The Wine Trials which are the foundation for the book of the same name which recommends wines to consumers. This is hardly a paper summarizing an unbiased "scientific" academic exploration.

Basically the book concludes "our studies show that critics can't be trusted but we can (be trusted)."

[Darren72]

Ironically, I don't think the paper shows this at all.

Could you be more specific? What part of the paper are you referring to that contradicts the abstract quoted above?

[JohnL]

Ironically, I don't think the paper shows this at all.

Could you be more specific? What part of the paper are you referring to that contradicts the abstract quoted above?

←

Unfortunately, one needs to access the many references in this study to fully understand its failings/short comings.

RL Weil and especially Goldstein whose "tests" form the raison d'etre for the paper.

The methodology used criteria based upon the assumption that only 90 point of higher wines represent recommendations by professional critics. This is simply false based upon the scoring system explanations/criteria of the critics.

In fact, as I noted many of the wines Goldstein's test reveals as preferred by the subjects have received positive recommendations by the critics he claims. Thus any conclusion that "critics are fairly poor guides..." is wrong. Goldstein actually proves this.

It is significant that Goldstein et al do not reveal key raw data such as a list of the wines the testing included that formed the higher price category. In fact, they do not reveal most of the wines used in the test let alone any breakout of how participants responded and what each tasting flight was comprised.

What they do is rely heavily on a lot of "other" research and some arcane formulas and statistical argy bargy.