The only sidequest currents that take more than a couple of minutes are the two that require optional dungeons (Dusk Vigil and Shisui of the Violet Tides). And that's only really a big slowdown if you're a DPS and have to wait in the queue.