If the paint is coming off, the paper drywall covering may be coming off. If so, I would suggest trying to scrape off any of the loose, fuzzy fibers without damaging the drywall. Possibly sandpaper.
Lay down a nice skim coat of drywall mud (top coat), whether only the spots that need it and feathering out when it’s dry, or skim coat the entire surface.
Primer is a must imho.
If you plan to re-apply wallpaper, be sure to use "sizing", which can be found at any big box store, Sherwin Williams etc.
If you want it done right, guess you gotta do it yourself. :D