Thwe sandy grain will stay somewhat, but after you put a clearcoat on, it won't feel gritty.
Don't scuff it before the clearcoat. It'll stick just fine.
It may smell like "paint" for a couple of days, but it goes away.
Sounds like you are doing it right.
Post pics when you are done.
Barry