It sounds like a case of neglect.
No worming, no hoof triming, etc.
If it were me, I'd take him to a vet THAT KNOWS SOMETHING ABOUT GOATS and have him looked at. Write down the backstory for the vet's information and see if you can get a diagnosis.
Take a sample of 6 nannieberries from him...