Depends on how long they're in there and how much more space they will have.
I'm going off of sheep space requirements--I assume goats would be similar.
If the goats have access to a pen/pasture/lot outside the barn, the goats only need 12 sq ft/hd of space inside their barn.
If the goats have only this pen as their housing, I would do at least 30 sq ft/hd.
My five sheep share an 8x8 shelter, it gives them 12.8 sq ft of space each, and they can comfortably lay down in there all at once, with room to spare. And my sheep are bigger than your average pygmy--about 24-27" tall and 150-200 lbs (except the two lambs, they're about 80 lbs a piece).