Selected dungeons and guildhests seem to take much longer. My guess is that it's down to the roulette feature. The roulette seems to favour a select few so that depletes the pool for those dungeons not on the roulette rotation.