If the concrete surface is smooth enough, you could try using a razor scraper. Worked like a charm when I had a similar situation to yours a few years ago. Bought the scraper I used from a DIY flooring place. It has a 3' handle with a stout vinyl grip, meaning you stand while using it. And it uses replaceable blades, which are about 4" wide and quite strong. Any stubborn residual glue could be wire-brushed off with a little help from a dab of lacquer thinner.