If your looking to make it perfect then that would be fine. Any old house like that is going to have a rough surface because almost no one takes the time to remove every speck of paint.
I'd use a citrus or soy paint remover. Non toxic and will not burn your hands, eyes and plants, It also encapsulates the lead paint that most likly was on there that you have been breathing.
If you want to never have to paint it agin I'd get the window frames wrapped with PVC coil stock and get replacement windows if you do not have them now.