This question pops up quite regularly. It's all about how to hold the piece while you paint it and set it down to dry. As Patrick says, lots of parts will need to be built before painting so no problem with those. For tiny parts and parts you may want a different colour to the main body of the model, there are lots of options open to you depending on your personal preference.
Painting on the sprue gives you plenty of options where and how to hold the piece - you can cut a section of sprue holding the piece you want away from the main sprue as desired - but cleaning up any flash and mould seams can be tricky. The other disadvantage is that when you cut the piece away from the sprue and clean up the attachment point, you'll end up with an area that's not painted - and 99.9% of the time this WILL be a part of the piece that will show when you fix it to your model - meaning of course you've got to get the paint out again to touch up the missing area.
The other option is to cut the piece away from the sprue, clean up the attachment point and any flash or seam lines, then paint. This obviously means that you shouldn't need to touch up any of the paint when you come to fix it to your model. The disadvantage is that now you've probably not got any way to hold the piece while you paint it. You could hold it and paint half the piece, then when the paint is dry, hold the bit you've just painted and paint the other half. That's fiddly and messy and you've still got to put it down without the new paint touching anything else.
I suspect most of us cut away the piece and clean it up and maybe build it into sub-assemblies, then hold it using small clamps on attachment stubs, sticky tape, bluetack, a cocktail stick stuck into a hole or even glue it temporarily to a stick (if you opt for this one, DON'T use your normal plastic cement. Use superglue or PVA so the piece will break away from your holding stick easily). What you need is some way of holding the piece firmly enough for you to paint it.
Pick a place to hold the piece which will not show when it is fitted to the model so no touch up is required. Sometimes you will need to think of the best way to hold the part and many ingenious methods have been used depending on the size and shape of the piece and where it will be fixed to the model.
Don't forget that you will need to put the piece down to dry when it's been painted. Using cocktail sticks with the part fixed to one end seems to be favourite as you just stick the other end of the cocktail stick into a piece of polystyrene, or hold the other end in a clamp. I've seen guys use the pegging boards used for cribbage to hold rows of cocktail sticks.
One other advantage of taking parts off sprues is that you can gather together all the parts needing the same colour and paint them in one lot - and you also avoid the problem of masking the other bits on the sprue if you spray paint them.
My apologies if this is long-winded - I've just expanded on what Patrick and Moni said, but it seems now I have some free time on my hands .........
