Shooting stuff has nothing to do with winning a beauty contest! If the Hogue feels better to you than you will most likely shoot more & therefore better with it. So do it! I bet that camo stock would fetch a couple of bucks on PM classifieds to offset the cost of the Hogue stock too, if not cover it altogether. Green on green camo would probably look kinda cool anyhow...IMHO.
For reference, I sold off a beautiful Richard's laminated wood stock on my 10/17Mach2 to put a Hogue overmolded stock on instead. As in your case, the Hogue felt much better to me when out hunting squirrels & bunnies on real cold winter days. The laminated stock would slip through my gloved hands too easily.
Although it doesn't look as pretty, I love that little rimfire now more than ever with the Hogue!
good luck...