There is another factor, which has been overlooked in this discussion. SE penchant desire to allow for oversubscription to the servers. Gilgamesh and Balmung (more so in the past) have constantly been allowed to have too many people registered on the server beyond the server's capacity to serve that many people.
Think of SE like an airline that consistently oversells its flights. Yes, SE did provide a brief window during which people were allowed to change to a new server free of cost. Unfortunately, many, many, many players returned to the game after that window ended and yet still before the SB2 launch. This increase in population has pushed the server beyond it's capacity. The real fix is to offer people the transfer window closer to the launch period. Thus, providing a much larger population with the option.

The world transfer service has not helped matters, as wanderers are consuming space on the server and denying access to homeworld players. Yes, wanderer priority is lower when attempting to transfer to the server but nothing is done about them once they are on the server.

OP, as a fellow player suffering from these awful queue times the best advice I can give is, adjust your life schedule as much as possible. I start the login process at 7:30 pm EST and hope to be on the game within 2 hours. I have also started waking up an hour early in the morning when the server load is very small and there are no login queue times.