Wow...
My experience is the exact opposite.
I took 42 from pastures last year alone.
In fact, almost all of my groundhog hunting is dedicated to pastures, because I get the permission from farmers to go eradicate the little ground grizzlies from the pastures so cows won't step in the holes.
Now, I don't usually find the holes in the middle of the pasture fields, though I do occasionally. Most of the holes are on the field edges, near wood lines, creeks, fencerows, etc. Somewhere that offers the little piggies cover and concealment for themselves and their burrows, as well as access to the pasture fields and edges where there is plentiful grazing.
Of course, this may be a mid-Atlantic/mountain area thing, but I'd say it'd certainly be worth your time and effort to check, and offering to check, and then CAREFULLY clear those pastures of groundhogs for local farmers is an excellent way of accomplishing two very important things: 1) gaining access to new hunting areas, and (perhaps most importantly) 2) doing a good service as a hunter that can and perhaps will better our image.
Good luck, and good hunting.
Oh, btw - for pasture duty, a good .22LR or small capacity .22 centerfire (I use a .221 Fireball) is about perfect. Plenty of punch for the generally shorter ranges, and not enough report to spook cattle, farmers, and neighbors.