mushrooms are colonial organisms that, once established, are very difficult to get rid of. they make fuzzy white colonies under ground called mycelium. that is the real bugger, not the caps and stems you're seeing pop above the ground. those are basically just sex organs. if you want to stop them, you've got to kill the mycelium. it (they) can live for centuries and spread like the plague with the right food source.
your best bet is to go to your local grow shop, get a fungicide, and get at the colony. dig or drill narrow holes in the colony area and administer the fungicide below the ground as well as on the mushrooms that you can see.
that should take care of it.