I don't know what your actual issue is but close in zeros, such as 25 yards, 50 yards, etc., means the bullet will continue to rise due to the super elevation of the barrel, as mentioned by Kirsch, and the rapid intersection of the bullet's path and the aiming devices line of sight.
The closer the INITIAL intersection (in your case 50 yards), the higher the bullet will rise. Eventually, the bullet will come back down through the line of sight creating two zeros, a NEAR zero and a FAR zero. The max rise in that established arc between the NEAR and FAR zero is called the Max Ordinate, an artillery term.
The only thing I can think of that would create a windage drift such as yours is the aiming device's line of sight is not aligned with the rifle's bore. Any offset, no matter how minor, will be exaggerated at distance, just like when using a laser aiming devise that's offset to the bore.
Seems to me a 50 yard zero should be slightly high at 125 and darn near back on at 200-225. A ballistics chart and field testing will give you SPECIFICS.
