Making an encounter random does not make it harder.
In fact its the opposite, sometimes it becomes harder, sometimes it becomes significantly easier.
This just leads to benefiting people who have a lot more spare time (to pull until they get a lucky easy pull where things lined up). It also makes it far harder to tune as tight as they do. The only way to make it seem harder when an encounter is random is to increase the raid size. However the difficulty from that is artificial. As in the chances of someone making a mistake goes up exponentially with party size.

The raids are fine and well tuned on release. If you can't beat them in a reasonable time after they are released you are bad, or you don't have a good group, or you have little time to play the game.

You also truly don't have to memorize entire fights to beat them. You just have to learn counters to specific mechanics, which if the order was random my group would still beat, easily, and you still wouldn't although your excuse and complaint would have to change.