Well, I'm not an expert, but I can tell you what I have found with my 25-06 - a Browning Composite Stalker.
I have tried a variety of bullets and two powders. I shot 120 gr. Remington Core Locks and IMR-4350 to 1 1/4" 3-shot groups. That was the norm, give or take a little.
Then I tried both Sierra and Hornady 117 gr. Both loads were shooting about 1" to 1 1/4", again, that's 3- shot groups. I used IMR 4350 on the Sierra and H4831 with the Hornady.
I read an article one time by the late Bob Milek, wherein he talked about H4831 and the 25-06. He favored H4831. I had been experimenting with IMR-4350.
Finally I went to a lighter bullet; a Speer 100 gr., and BINGO ! Using those bullets and H4831, it will shoot consistently between 1/2", 5/8" and definitely under 3/4" groups all day long. Again, that's 3-shot groups.
Probably you will need to just do some time at the reloading and shooting benches. After some experimenting, you will most likely find that pet load that your rifle prefers. Seems that it already likes 75 gr. bullets, so you might want to give a 100 gr. a try. The fact that you can shoot 1/2" groups with the 75 gr. bullets seems to indicate that the rifle is capable of good groups. It may favor lighter bullets.
As to 100 gr. Speer performance; well, that's all I use now. I have taken 5 Wyoming antelope with that load, the last one being a 300 yard shot. The bullet has performed great. I have also taken a mule deer with that load, a Corsican ram, not to mention various crows and groundhogs, all with the 100 gr. bullet. Some may say that bullet is too light. I say whatever works, works. It works for me and puts deer and antelope size animals down without a problem.
I treated my brass with care, going the extra mile you might say, in hopes of obtaining accuracy. Seems that it paid off. First, I hand-weighed all cases and separated them into groups of like-weight, with none being more than .3 of grain difference, most less than that. Then, I deburred the flash hole in the case. I also turned the necks very slightly, and I use Lee Collet dies to neck-size. I hand-weigh each powder charge. That may not be necessary, but I do it with all my loads anyway.
I can't argue with the results. It's a great rifle / load combination, and I love it.
Just stay after it and you will find what your gun likes. If, after quite a lot of time and effort, it still won't perform, then you have the option of trying other things, such as free-floating the barrel, or recrowning. If that doesn't work, you can always rebarrel, or help it to find a new home. I'm not convinced that these things are your problems, though, again since your rifle will shoot 75 gr. bullets well.
I had a Ruger .257 Roberts one time that would just not perform the way it should, and I really wanted to make that rifle shoot, because I liked it. I tried everything from various powders, bullet brands and weights, free-floating, changed to a synthetic stock, glass bedded it, recrowned - well, you get the idea. It found a new home since it just would not peform the way I thought it should. I could have rebarreled it, but by that time, after all that effort and expense, I was quite worn out with it and just wanted it out of my sight. Besides, rebarreling doesn't come cheap. In hindsight, I could have probably rebarreled it as cheaply as I did all the other stuff.
Sorry if I rambled on too much. Just thought I would share my own experiences. Hopefully that has given you more to think about.
Good shooting and keep us posted.
