Thanks for the advice guys! I'll try annealing and see if that fixes the problem. I'll also investigate changing my setup, I may have bit off something rotten and too much of it.
On a different note, I did end up with a load today that is consistently .5-.6 MOA at 400 yards. My frustration is that the rifle seems to shoot everything I put in it about MOA, but it really takes some tweaking to get it down to .5 MOA. Like 200 rounds of tweaking. And this is a new Benchmark barrel. Maybe you're right Furhunter, and the dies are the root of my problem.