Accuracy is usually not the main issue when using light bullets in a very fast twist rifle.
The bigger concerns are A) Since light, high velocity, varmint style bullets generally have very thin jackets, the high rate of spin imparted from a 1:7 barrel can cause the bullet to disintegrate in flight. And B) Fast twist barrels sacrifice some velocity compared to slow twist barrels which are specifically designed to optimize velocity from lighter bullets (energy being used to generate high spin rate instead of forward velocity).
All that being said, if you have a high quality fast twist barrel, you should still be able to get decent accuracy and velocity from lighter bullets so long as you don't try to push them too hard. Trial and error will tell.