Brick is an OK insulator, as long as it is dry.
However, you need to look at this another way.
For a fire to burn well on tick over, you need a warm chimney, from this point of view you should encase the chimneys in at least 2 inches of polystyrene to keep the heat in the chimney.
At the same time you want to reduce your heating costs.
Again by keeping the heat in the rooms/chimneys you reduce costs.
Several inches of closed cell insulation will pay for themselves in a year or two of saving on heating costs.
Five inches of polystyrene will reduce your heat loss by about 80%