don't know if it would be cheaper than a modular home or not, but if you are going to do it you should seriously look into installing floor heat when you do it, you will not regret it. as far as modular homes go you may want to look around as they are pretty nice these days. My MIL used to work for Bonnevilla Homes out of Aurora NE, and they shipped homes all over the states. I was very impressed with the quality of the homes.
BTW, my uncles put up a 90'x 70' with floor heat in the concrete, hired all the labor except the dirt work and had it built for around 100k, they built a bathroom and an office in one corner, the rest became the farm shop. So your Idea may be doable.