Modulars are just fine if you get what you pay for. As with any home getting a product you paid for is the issue.
What happened in my experience was that the home was built fine but was damaged in transport, or rained on when not completed, then installed anyway. When the fiberglass insulation gets wet, it no longer has the r-value it once had, and the home gets sealed up without drying out correctly because it needs to be finished.
A home is what you call it when there is a roof over your head, just be aware of any pitfalls and you will be OK.
So go modular if you can, it saves time and money and can be a superior product to other style homes. I have seen some stick built homes which should be used for fire department drills.