@Legamin I recently had to sell my horses due to unfortunate circumstances, but I'm keeping my sheep and dogs. Moving to a temporary place until I find my next farm. That said, I understand the small livestock versus large livestock. My horses cut a deep trail down a slope that rain and erosion washed out even worse. I had to fill it in, but they still walked in the same place. I dry lotted them frequently (same with sheep) due to new and limited grass to keep them from killing it.
You will have to have a frank discussion with your neighbor and explain the damage the horses do to the pasture. The weed seed distribution alone would be enough for me. If he looks at a destroyed pasture and sees no problem, then he needs to destroy his own pasture, then scratch his head or maybe his butt, same difference, and wonder where his grass went, and why there are deep ruts eroding his soil away. Tell him that you will no longer be able to use weed killer and he probably doesn't want to come dig up every weed clump his horses poop out.
Or even better, buy a steer for the freezer. Use the excess pasture to feed the family. Use hot wire to contain the steer where you want the weeds eaten or trampled, move him often. In a tight pen, the steer will have to eat or trample the weeds. Trampling crushes the weeds down where worms and soil microbes can break down the humus and improve the soil. Leaving the steer in one place until it is a barren dust or mud hole is defeating the purpose. Extra work, moving daily, but it sure helps a pasture.
Either way, you don't need to let the horses come back. Bringing 6 horses to graze your small pasture is inconsiderate and overkill. If you must bring the horses back, limit it to 2 and explain why. Don't be surprised if he doesn't get it.
If nothing else, mow it. A mulching mower will chop up the grass/weeds and return the humus to the soil. Better than horse damage.
Don't get me wrong. I love horses. It hurt to have to sell mine. But I'm going from 8 acres to 2 acres, only about 1 acre for the sheep. No room for horses. When I find what I want, I want horses again.
You can actually use horses or cattle as dead end hosts for parasites. Sheep parasites are different from horse or cow parasites. Rotational grazing with different species can curtail parasites for the different species. You can use electric fencing to divide the pastures, easy to put up and take down.
Parasite larva crawl up the blades of grass, waiting for the animal to bite it off and continue the parasite cycle. But if a horse or cow bites it off first, the sheep parasites don't survive. It works in reverse too. You have to move them often to prevent damage, wait for the grass to regrow, then graze the sheep. Something to think about.