Its not my cup of tea

"its not my cup of tea" is the disclaimer of choice when we decline an offer , beg off from an activity or proclaim a dislike for something. This is fine as long as you are a tea aficionado. What if you are not? Could a coffee lover get away with its not my cup of java ? If coffee doesn't do it would its not my brand of scotch suffice? Neither of these alternatives would cut it. They are not clichés and to shorten conversations or make a point, one must speak in accepted clichés even if they are a misrepresentation of taste.