This happens to me often during "peak" hours at Central Standard Time. It's more to do with my internet provider than FFXIV, though. I'll "idle" according my FC members on team speak for a few seconds, then the game "catches" up in very fast sequence. My internet provider is fine for the most part, but I have limited options where I live (more cattle and corn/beans than people).