To clarify, the two 40 amp that are tied together turn off most of the house but when they are on there are other 15 & 20 amp breakers that turn off different areas that are controlled by the two 40 amp.
I'm not sure of the wire size coming out of the two 40 amp. I see other wires that say 4 AWG and it seem to be a little larger than that. But I see that the wires coming out of the two 40 amp go behind the breakers and appear to be very short. Un-like all the 15 & 20 and the one 50 amp which all go out the top of the box into the attic.
I do have another breaker box next to this one that has one large main switch but I have never checked to see if it turns of every thing I assume it does. I will check once it is convenient for every one in the house. It may just turn of this box. This box has larger amp breakers 30's & 50's for water heater, range, hot tub-(no longer connected).