Checking the condenser unit, the only electrical connections I can access is a box mounted on the house, which has a plastic plug with 4 connectors on it. I'm not sure how to check that for power so I decided not to poke around it.
I tried turning on the heat again, the unit inside the garage started and the thermostat started clicking again. It read aux heat, there was air coming out of the vents but it wasn't warm. The condenser unit was not running. After leaving it for about 5 minutes, I come back and that thermostat has no display, the air is no longer coming out of the vents either. I checked all the breakers, nothing was tripped... reset them anyway. Still no display on that thermostat, I even swapped thermostats, still no display.
By the way, I have 2 of each units... the one I'm having troubles with is the master bedroom one. The rest of the house which uses the other condenser and larger garage unit works fine.