I don't think it's really a castration thing per se. If you remove the testicles, urethra, and penile sheath, you significantly reduce the potential of nasty excretions getting on the meat while skinning/gutting the critter. Whenever possible, I like to wash the entire hog (with the skin on), then put the hog on his back in a skinning rack (those Texans know some stuff about skinning/gutting), and start skinning. I start low and leave all the parts I mentioned above attached to the hide. Skin them right off as soon as possible, then continue skinning and gutting.
For what it's worth, I do the same thing with the "nipple" gland on the back of javelina. Get those funky smelling excretions away from the meat!