The delay is so people don't swap between jobs in the middle of a fight, and there is no delay when you switch in town.

I've had glitches before where I had a delay when swapping to a DoH job in town. I had to wait about 60 seconds for Manipulation cool down.