If noise is an issue, you need something else altogether like a 22 hornet or that new 17 Hornady Hornet that is supposed to be 'coming soon' - a 20gr V-MAX's at 3,650 fps should be plenty for most ground-hog applications.
The 22-250 booms, whereas the 204 has a higher pitched bark that bothers my ears slightly more. Bottom line though, they are both loud, fast, flat and effective 300 yard guns!
When I was deciding this question for myself, I based my decision on several factors. I had shot 2 friends 204's, and I really couldn't stay on target through the recoil with either one, like is advertised. Both 204's also had to be handloaded to shoot really accurately, where I had always heard great things about the inherent accuracy of the 22-250 round and its legend as "king of varminting". (I have another buddy who would tell you the 220 swift is the real king, but that's another story. I wasn't interested in the extra barrel burn that comes with Swifts.)
IF I could have been guaranteed that the 204 I bought would shoot 39's really really well, I might have considered it but with the prospect of wanting to get into coyotes and have a bullet that wouldn't be as susceptible to wind, and where I just wanted more bullet selection as a reloader - yep, I decided to go 22-250, and no regrets! I load a 52gr Berger Match Target hollow point with Varget powder and it just drills. I'm building another 30-40 rounds right now in fact.
Stick with that 22-250. It's a classic!