I don't know about the goats, but my friend who has Angora rabbits says you can pluck or shear them. Plucking should only take off the hair that's ready to come out, you don't yank it all out. And she shears hers with scissors (carefully).
Yes, you can brush out loose hair and use that, too. It may just take a bit longer to get enough to work with. Learning to spin with a drop spindle isn't hard, is there a spinners/weavers guild anywhere in your area? Check with 4H groups, the local agricultural society, the folks that run the county fair, or try the SCA ( sca.org to find a local group), they often have members that do fiber stuff.
Sending it somewhere would be more expensive, and the mills around here have a minimum amount of fiber you have to send - I think one was five pounds minimum for turning it into yarn. Besides, working with your own fiber is fun!