Even a pony can really hurt your kids. IMHO, leave it to the experts. It will be money well spent. Go watch and learn if they will let you, and see if they will give you some "homework" and help you get it right. It takes many months to make a pony safe for kids, especially kids who are not already very accomplished equestrians.
Newbies do much better with older, well-trained horses and ponies. Something large enough for the adults to ride, too, would be great, since the animal would get more work. An inexperienced animal needs to be worked with almost every day for months, and very regularly for the first few years, to be safe for newbies.
I would never get anything younger than 8 for children, and I trained horses and children for years. There are rare exceptions, but again, those generally are for experienced adults and experiences kids. And there is still some danger. For example, I have put kids on 3-5 year-old horses, but only for a few minutes at the end of a long work session on a hot day when I'd already tired the horse out. And I closely observed and corrected the child often. And did it rarely, not daily. Every time a horse/pony is handled, it is being trained, for good or for bad.
Hate to sound harsh, but if I had a dime for every person who told me they were scared of horses because of an experience as a child.....alway, the story is something that was very avoidable and made me gasp in horror that an adult would put a child into that situation.....I'd be in good financial shape!
So it can be done, but you will not save money, you need to be very willing to spend it. I think you will have a much more enjoyable experience if you find a truly well-trained and well-behaved horse or pony, and have an experienced person help you with the purchase decision. Be sure to watch the animal being ridden first, then your expert friend, then if it is safe, have whoever the animal will be ridden by give it a try.
Then go back on another day, and try it again, this time before it is exhausted by all that "trying!" Lead it in from pasture, groom it, pick up it's feet, saddle it, ride it, hose it off, use clippers on it, etc.
Safety for the children first, and worth extra money, IMO.