A 3 coat plaster wall may be up to 7/8" thick and usually 1/4" wood lath behind that.
So you have assumed the the brick will be good looking when it is exposed and that it should be right behind the plaster.
Both those assumptions may be right but not for sure.
You are not there yet, you will start to see the morter lines between the brick.
When the built the chimney they built a rough in shape and then the finished what would be exposed, if this was never ment to be exposed, you will find the rough chimney with whatever was the cheaper brick of the day. If it was plastered right on the brick you will not find the wood lath.