What you have is a three way switch controlling the red wire at the fixture. This would probably have been for the light. It is not uncommon to have the light powered by switches and the fan always hot. The chains on the fan would be used to control it and the light could still switched off by the switches.
The unlabelled black box between sw2 and 'from panel' would be unswitched power to some other devices like receptacles or other light switches.