There are several ways of calibrating a thermometer, depending on how good a physicist you are or want to be!
Many do include a ice point calibration, but then you have to worry about whether the ice bath slush is exactly at the triple point, and how pure your water is. (You do make your ice from distilled water, don't you?)
Another possibility is the boiling point of water, which also involves how pure the water is, how rapidly the water is boiling vs. simmering, how much air is dissolved n the water, and of course your elevation and the atmospheric pressure that day. In general, I would say that a boiling water calibration is likely to be off by as much as + 2C, whereas a ice point calibration is more likely to be within 1C. But since thermometers in general, and ESPECIALLY digital thermometers, are like to suffer from linearity problems, it makes sense to calibrate them as close to the working temperature as possible.
The problem with both of those methods is the question of linearity, as well as the convenience issue, and what you are trying to measure.
An ovulation thermometer would have an upper limit of about 100F or maybe 38C, plus or minus a bit, but for sous vide work, that is very close to the working temperature. And for under $20 US, they are very accurate, and quite inexpensive.
Now, for melting sugar, or making candy, or checking the temperature of your oven or your Himalayan salt block or cast iron pan before searing something, obviously 38C is a bit too low, and something like an accurate IR probe would be better.
Of course, a laboratory instrument would be more accurate, and cover a significantly wider range. I love my Control Company Traceable 4000 reference thermometer, which is certified to be within 0.05C at 60.002C, but it cost nearly $400, and I am two years overdue in sending it back for factory recalibration. And Pedrog has a Swiss or German unit which offers equivalent accuracy, although I don't recall the manufacturer. Both offer a logging function via computer, which is almost indispensable when trying to measure heat flow.
But when I was working up some presentation material on sous vide, I calibrated over a dozen off-the-rack thermometers I had in my drawer and several purchased from various kitchen stores. The Taylor brand seemed particularly poor, with some being off by 5 to 6F. Most were within 1F, but only after re-calibration against a reference thermometer. Only the Component Design Northwest Q2-450 was rated (by me) as Highly Recommended, other than the ovulation thermometers and my reference. Of the rest, one was Recommended after Calibration, two were Acceptable after Calibration, two were Acceptable, one was Marginally Acceptable, and two were Not Recommended.
The standards I used, which admittedly could be debated, were that in order to be Highly Recommended it had to be accurate within 0.0F, 0.2F for Recommended, 0.6F to be Acceptable after Calibration, and 1F to be marginally Acceptable after Calibration. A standard digital fever thermometer was -0.4F at 99.884F, which I considered Acceptable, but Not Recommended as a Secondary Standard. In fact, I'm not sure I would consider it acceptable as a fever thermometer for my child!
I also have to concede that I am not publishing these results in Consumer's Reports, and I didn't test dozen of thermometers from multiple lots from each of the various manufacturers. It would certainly be nice if someone would do that, but I can't afford either the time or the money to do so! YMMV, in other words.
It is an unfortunately fact of life that most of the products that were in the lower range of acceptability were all made in China. Now, whether that represents sloppy standards by the Chinese manufacturers, or a tolerance for poorer quality vs. expense by the importer or brand name retailer, that's an interesting question. But the Highly Recommended ones were all in the $80 to $100 region. You get what you pay for, and then only if you are lucky AND calibrate them!
Now, the question of how much accuracy is really needed for sous vide work is a different issue.
If you look at the time/temperature guidelines published by PolyScience, or for that matter in Thomas Keller's "Under Pressure," to my mind they are all about 5 to 7C too high. Now, maybe that is just a function of the way I like my steaks -- my wife might not agree with me, and you might not either. And that also is a function of how you sear your steak afterwards -- I typically cook mine at 52C, and then sear a 35mm steak with a torch.
For fish, I think that 1 or 2C makes a significant difference. And as some of the discussion on cooking eggs shows, 1C makes a significant difference.
And for long time, low temperature cooking, you really want to be sure that you are above 55C, otherwise you might be running a bacterial incubator!
So in summary, I think that an absolute accuracy of 1F or 0.5C is essential, and better than that is highly desirable, considering the possibility of drift.
But you really can't calibrate a working thermometer to those standards unless you have a reference thermometer that is significantly more accurate, say 0.2F or 0.1C, and as my results show, you can't trust most of these products to be accurate without calibrating them.
Hence the need for a secondary reference thermometer, and the best I have found for the price have been ovulation thermometers. Buy a couple, from different manufacturers, just to be safe! My two ovulation thermometers are both accurate to 0.00F at 100, compared to my reference thermometer, so I trust all three of them.
Thank you for the extremely comprehensive explanation! I've copied this into my ongoing kitchen notes, since there's a fairish chance that a sous vide setup is in my (admittedly somewhat distant) future.
I'm hoping the Thermapen I'm waiting on will do the trick for now (at well over 100 C, the issue of bacterial growth is less of a concern, even if it turns out to not have quite the accuracy I seek), and it has simply got to be a a step up from the cold water test.
. . . .
4. A .01g scale. The booger with significant digits is that a scale that only has one decimal point will be between .05 and .15 grams when you want exactly 0.1 of something very unusual. Very small units rated for 200g cost about $16; I recently picked up an American Weigh scale no doubt designed for drug dealers that works very nicely indeed. (You can recalibrate them, too!)
Thanks; were did you get your scale? I haven't stumbled over anything in NYC these days, although I haven't done what could be described as a proper hunt for this yet, either.
Search for the AWS-100 Digital Scale. you will find it for about $10-$12. Add a 100 gram calibration weight for a few more $ and you are all set. I have had one for two years with never a problem.