The party's "governing system" must be a democracy, so if majority wishes to go slow/fast, minority should leave or oblige.

That being said, I don't think that going slower is bad per se. Some people like to take it chilly, or maybe are tired after work, or whatever else. They have right for it too, and both approaches to dungeoneering are equally fine.