My first inclination is to say it's not advisable. The stucco is probably applied to wire mesh, which in turn is fastened to the chimney. But even if it's just stuck directly to the brick, I can pretty much assure you won't get it off without doing some damage to the brick. Also, I doubt you would get all of it off.
On the other hand, if it was not applied properly and is coming off, it may be a completly different story. Maybe a hammer and wide masonry chissel might take it off without causeing too much damage to the brick.
If it is practical to remove it, you'd probably need to follow up with tuckpointing and an acid wash to remove the remaining residu the stucco is sure to leave behind. But like I said, I'd be more inclined to leave it alone.