I'm not aware of people widely criticizing him because of his appearance. Maybe they do, but as far as I can tell, the "hating" started when he opened his travesty of a restaurant, and when his shtick started getting old. I mean, even Emeril had a shelf life.
And sorry, in a foxhole I'd pick Bourdain over Fieri in a heartbeat.
Just my two cents.