First you need safety equipment, safety glasses and some sort of dust collector, even a shop vac is better than nothing.
Next is chisels, Harbor Freight has a set of HSS chisels, less than $50. A 4 jaw chuck, they start at about $60 for a decent cheap one. Mandrels, depending on where you buy and the quality, look at $40 to $90 a piece. Basic sharpening set up, hand stones with oil will get you by for less than $20, up to a Wolverine system with a cheap grinder adds up to over $200.
Sanding supplies, a good finish starts with sandpaper, 120 grit through at least 600 grit from Klingspor will run you about $40. Finishing supplies, a can of spar varnish, teak oil and a CA glue will cost you around $30 to $35. Wood, this can be cheap or expensive, at first look for the cheap stuff, it doesn't hurt as bad when you mess up a cheap piece of wood as it does a piece that cost $3 or $5 or more. I know I left things out, but this should get you started.
Back the the HF lathe, I would save my money, and get one that has 2MT and 1-8TPI in the headstock and 2MT in the tailstock. These are the standards for most good mini lathes. You may not need it to start, but in the long run it will save you money on accessories.
