If you're on a budget, get the Lee Anniversary reloading Kit. If you can spring for it, get the RockChucker kit. I'm still using my Lee kit and it works well, but I would have the Rockchucker kit if I had it to do over again.
Either way, buy the Lee Deluxe Rifle Die Set and a Factory Crimp Die. The collet design Lee uses in his dies is sweet.
For case trimming, I recommend the Lee Zip Trim ($15) and a Lee case length gage and lock stud, trimmer, primer pocket reamer, and chamfer tool. A primer pocket deburring tool is also a good investment, especially if driving tacks is in your future. If you get some case cleaning solution you can economically clean your cases with a rag and the cleaner on the Zip Trim. I have a tumbler, but that may be a big expense for a beginner. If you full-length size (I full-length size all my hunting rounds so they chamber easily), you DEFINITELY want clean brass so you don't scratch your dies.
Case Lube - I use the Lee case-lube because you can wipe it off easily with a rag. Other lubes may change the combustion performance of the powder and you have to make sure to remove 100% of the lube before priming and charging.
For starting loads, I took my favorite factory round and measured the overall length. I then looked at the powder to see if it was a ball-type or extruded powder, and used the same or closest available projectile. Use your reloading manual data to select the charges. Start with lighter loads and watch for pressure. A good reloading manual will tell you how to recognize symptoms of high pressure.
Definitely get a complete reloading manual (I second the Lee recommendation) so you can learn the entire reloading process. I also bought a book of caliber-specific load data for my .22-250 that contains ballistic data from all the major bullet and powder manufacturers.