To me, the whole varmint/predator rifle thing is all about 22 and under cartridges. I know that many use 6mm and .257 cartridges with great success and that's fine. However, once you get much over 25 caliber you are leaving, at least IMO, the realm of varmint/predator rifles and are entering into the big game domain. It's purely subjective; I'll give you that.
Anyway, more power to you if you want to go the 308 route. You shouldn't have much trouble finding an accurate load with that round.
As far as disadvantages are concerned, if you care anything about fur, you are not exactly picking the best option. Other than that, have at it.
Landrum