I get this too, sometimes the recast can get as worse as 6-7 minutes long and it makes tanking really annoying. When starting a guildleve, the recasts are all reset and if they're all out of sync, I can't use any of my abilities for several minutes.