Getting high quality fruit from a backyard tree can be quite a lot of work. You would need to begin spraying the tree with a fungicide such as Captan or Funginex in early spring when the buds begin to soften and "swell" before opening up. Then, when about 95% of the blossoms have dropped off the tree (petal fall), you would need to begin mixing an insecticide such as Sevin or Imidan with the fungicide and spray the tree every 10 to 14 days throughout the season. You never want to spray insecticides while the tree is in bloom or you will kill the honeybees needed for pollination.
There is a product called "home orchard spray" which has both fungicide and insecticide in it and this could be used starting at petal fall. Spray whenever you get around to it and you will get some decent fruit and some with worms and apple scab.