Dungeon design evolved around the way players in random groups chose to play the earlier dungeons. Players would ignore the side rooms, they would pick a best route, etc.

Why waste time on adding the additional elements that would end up ignored in the long run?

We'll be getting the first Variant dungeon in a few weeks. I'm sure SE will be watching to see what players do over time. I'll lay odds that once all the possible combinations have been explored, players in random groups will once again determine a best route and expect everyone to stick to it.