Drywall is not structural. Too much weight on the anchors and they will pull out. SUCKS, right? The brackets shot into the studs probably did not pull out. You're facing a complete removal, wall patch/repaint, then going for it again.
First, pick up as many studs as you can.
Second, use at...