Aaronestes:
I agree with the other posters that you'd likely regret putting expanding foam insulation in there because you'd be likely to get it all over everything, including on your refinished hardwood floor. It's a fight working with that stuff.
What you DO want to do is get something thin and maneuverable, like a hack saw blade, and a strong vaccuum cleaner and vaccuum the dust out of those gaps as well as you can. The reason why is that dust is 90 percent organic matter, and consists of dead skin cells, cloth and paper fibers, plant pollen and road grit that gets carried into those inaccessible crevices by air currents. Bugs like silverfish eat that stuff.
It's easy to remove dust from the readily accessible areas of your house, but now that you have access to some inaccessible areas, take the time to remove the dust there so that your house will have less dust in it and therefore can't support as many bugs. Some bugs feed on other bugs, so the fewer bugs you have, the fewer spider webs you'll have in your house as well.
What you might want to do is buy something called "Foam Backer Rod" which comes in diameters in 1/8 inch increments from 1/4 inch to 1 inch. You should be able to buy foam backer rod at your local Home Depot or Lowes, but you're paying for the packaging in retail stores. If you go to any of the places listed in your Yellow Pages phone directory under "Caulking and Caulking Supplies" or "Construction Materials", many of them will likely sell/give you as much foam backer rod as you need if you contribute $5 toward their Staff Christmas Party Fund. (It's quite inexpensive.)
You use the diameter of foam backer rod that's just a little larger than the gap you want to fill. So, if the gap under your wall is 1/4 inch, stuff 3/8 inch diameter foam backer rod into it. Not only will it push in easily, you can remove it easily should you ever need/want to.
After removing the dust, stuff foam backer rod under your walls. It'll prevent drafts, insulate a bit and keep the outdoor bugs from consorting with the indoor bugs of your house.
Or, if it wuz me, I'd be more inclined to use foam backer rod than either expanding foam or caulk.