Because they said you wouldn't be able to?

Going to have to wait til your party is finished and out of the dungeon.