Are you sure the holes aren't weep holes, and intentionally put there to allow any moisture getting between the glass blocks to escape? Although I have very little experience installing glass blocks, I vaguely recall reading someplace that some systems incorporate weep holes near the bottom of the layout to intentionally drain moisture.
"Just had a glass wall built" implies you paid someone to install the wall. Why not give them a call, and ask them to come back and make good on what they did? I for sure wouldn't use any kind of caulk to seal the holes, as it may stand out like a sore thumb. Also, messing with things yourself will quite likely void any warranty that came with the installation, if that's a concern.