No one simply wants the fights to take longer, we just want the average encounter (As in, non NM and boss fights) to have more depth and complexity than they currently do.
The thing is, any such depth and complexity is artificial. No fight with any common rabble-rouser enemy should take more than 10-20 seconds, period. you swing your sword and cut his heart out, or burn him to ash with a fire spell, or whatever. He's not standing after that. Either that or he does the same thing to you and you're dead.

I'd like to see a bit more risk without making fights longer. I'm suprised that I have to go 5 levels up or more to get an enemy that's even a little threatening. Then one or two levels more, and it's an impossible encounter. There's something really out of whack with stat scaling in this game. It's not as bad as when the game first came out though, I'll say that much.