Try different things till you find what they really like. They are "prey animals" so are naturally skittish about and around things they don't know/understand. Mine love animal crackers (cookies). Some will eat raisins while others don't care for them. They pretty much all go crazy for fruit loop cereal mixed with honey nut cheerios or captn' crunch. It's sweet and grain based, what's not to like? Try to avoid a lot of sugars if you can... It's not really good for them and can upset their stomach/rumen. Remember. we're talking treats here, not everyday meals. Especially with the young ones, you want them getting quality fare vice garbage treats.
Try taking a book and go sit in their area and read. Goats, even being prey, are extremely inquisitive. They just HAVE to check out everything, even if it might kill them. since they don't have hands, this is generally done with their mouths. They need things to keep their little brains occupied. If they get bored, they can become destructive. There's lots of threads here you can go read to see what others have suggested and what they've found to work. Enjoy your goats! They can be a lot of fun. and please consider sharing some pics... we all love pics here!