You will need to clean all the old stuff off first.
Use a putty knife or wire brush, but don't get to rough with the brick, it needs it's hard candy shell exterior.

That's the easiest way to explain it.
Then go to a lumber yard and try to find "Geocell" brand paintable caulking. Kind of like GE silicone but better. But any caulk attached to brick needs to move with the brick, and I found this type to be one of the best.
Temporaraly install the vent and make some pencil marks as to where it will go. Then put some blue painters tape about a half inch away from these marks to keep excess caulk off the brick as you install it.
Just fillin caulk behind the vent and push it in, use the putty knife to remove extra caulk. Remove tape.
You should have some counterflashing installed over the top flange of the vent.
Copper.org: Architecture Design Handbook: Flashings and Copings - Counterflashing
However that is up to you. The caulk will need checking from time to time if you do not.
Good luck.