It's not random. Items are sorted first by per-item price. When multiple listings have the same price, they're sorted by stack size. When multiple listings have both the same price and the same stack size I believe it's the region it's being sold from (and therefore whether there's tax added) that's the third criterion. If all of those are equal, then it goes by when the item was listed.
Those first three criteria are about making it easier for the buyer to find the listing they want. The final one is about fairness to multiple sellers offering the same thing. Seems a pretty good arrangement overall.