I'm not trying to say anyones way is right or wrong, this is just what I believe from my own experiences using both red, & white light. I hunt public, & private land.
Intensity of the light is the main factor as others have already mentioned not the color used. If you have a very powerful red light (high intensity) it will scare animals at the same rate as a white light of equal intensity. If your red light is dimmed enough to not scare animals, then take the red filter off and reduce the white light so that the white intensity is equal to that of the light with the red filter on it, results should be similar with either setup. I believe the dimmer the light no matter the color is the key to avoid spooking animals. I keep mine so dim that I can't even tell that it's on unless I shine it on an object or the ground at about 30 yds or less, but believe me it will pick up eyes at great distances, & the animals never show any signs of spooking. When they get in close I brighten the intensity of the light just enough to identify & shoot, they do show signs of spooking as the lignt gets brighter, regardless of color.
Test out what I've said above by leashing your yard dog to something, so that he won't be able to follow you. Turn your light on him at about 30 yds, and dim your light down regardless of color so that you can barely make out the outline of his body. Now walk backwards, & see how far you can detect his eyes, it will amaze you when you get so far out that you can't even tell where your shining your light but, the dogs eyes will still light up like fire when you scan across them. Notice the position of your dimmer, & simply set it in that position when scanning, turn it up to identify & shoot. If you hunt open ground you may want your initial settings to be a little higher. If you will use your light like this regardless of color, all you'll have to worry about spooking the varmint is your scent.