Those are "weeps".
They are necessary to drain the moisture out of the space between the brick and the wall behind it. There are several options to close them off to keep the bugs out. One is to place short pieces of Oakum in the holes with the backside turned up, cover them with mortar and pack it in. Oakum will swell and freeze and swell some more though, which may break the mortar back out. Another option is to buy metal louvered weeps at the brickyard and install them. This is very labor intensive, but I have seen some creative ways to fudge the install. Cut the flange off of the top and glue the weep in the space with clear silicone caulking. It beats nothing, or plugging the walls.
The cold air coming out is puzzling. Could just be the cooler ground level air moving through the spaces.
The most air tight fix would be the oakum pieces. 6" long with the end turned up against the wall of the house and the other end at the bottom of the hole.
Welcome to the forum Carol!