Shim and trim on the trigger. The rough long pull is causing that much POA deviation is never going to be conducive to accurate shooting. Not using a bi-pod is a Band-Aid fix to a bigger problem. Either bed and float the factory stock or, better yet, replace the stock entirely. You should be able to shoot the rifle how you want and not have to dance around the issues. Fix the issues as a whole versus quick fixes that do not address the entirety of the issue.
I would also swap in another known-good scope, just to eliminate that possibility.
