First things first
First - Seems like you have two types of moisture problems in your basement. You are getting moisture from condensation and moisture from water infiltrating the concrete walls.
The dehumidifier might get rid of some humidity in the air but can't do much about the water seeping in through the walls. That needs to be addressed otherwise the water will evaporate inside the basement and your dehumidifier will not be able to keep up with the load.
I'd suggest an interior perimeter drainage system installed along the internal basement wall, to intercept water seeping in through the walls and the junction between the wall and the floor, diverting it to a sump pump system.
Since you seem to have a big condensation problem, a vapor barrier might be recommended. Waterproofing paints and sealants are not the best idea because they will not handle hydrostatic pressure and will eventually peel off.
The idea is to install a vapor barrier that will allow the water from the wall behind it to be diverted to the perimeter drain and keep it from evaporating into your basement. There are many products in the market that will do just that. From thick 20mil plastic liners to wall systems.
The energy savings alone will pay for the repairs overtime.
Second - Musty smells usually indicate the presence of mold. Even if you can't see it, it must be lurking somewhere. And my bet is... the insulation. Because you have all that moisture in the air, your insulation is probably damp and, if there is anything organic in it (such as paper) mold is feeding on it... Or on wooden floor joists.
I'd also take a look around that little blown hole under the dishwasher, if it smells musty as the basement. A plumbing leak or even rain water might be soaking the wall or insulation behind it... and since those tend to be organic...water + organic matter = mold.
I am thinking mold because it is a foul smelling fungus.If it it were something else, like a dead rat, urine or feces, it would smell much worse. It would not be something you'd describe as musty.