Time delay/cooldown when switching from DoH/DoL to DoW

Discussed in beta (can't find link).

If I am fishing (or mining/logging/harvesting) and I see a fate come up within a few meters of me, there's no reason that I should be forced to go into stealth mode [I was first here dangit! my fish!]
Though seriously, the delay is horrible when you want to join a fate or even save someone from dying on a mob that's chasing them, but you can't because you're standing there either with your rod in your hand (literally), or you switched to a DoW and you can't do anything for ten + seconds.

It really kills not only flow of the job swap system, yet also ruins immersion as it's a game breaker.

Can we not have delay when switching from DoH/DoL to a DoW (and vice versa) to be removed?

The delay is so people don't swap between jobs in the middle of a fight, and there is no delay when you switch in town.

I've had glitches before where I had a delay when swapping to a DoH job in town. I had to wait about 60 seconds for Manipulation cool down.