If you are discreet about it, it should be OK. It's the idiots with the DSLRs and long lenses who stand on chairs to get the "best angle" that should be shot. I have three DSLRs and several lenses, but only ever take restaurant shots using my cell phone (without flash).
Respect other customers and your guests and I don't see a problem.