The toilet connected to the water heater may have been done to prevent condensation on the outside of the tank. Instead of cold water filling the tank and condensation forming and dripping all over the floor, the tank fills with warm, maybe hot water and then the water cools to room temperature with no condensation on the outside of the tank. You are in the humid South..
As far as your pipes bumping and thumping, maybe you need an expansion tank? Most newer homes have them installed near the water heater. It gives the fresh water a little room to expand without causing movement or excessive pressure on the pipes, also, if there is a check valve between you and the main supply, you need an expansion tank anyway. There is no relief for the excessive pressure.