I would not group with them if they were to travel slowly? Correct.
Nor would they group with me if I were to play in a way they did not enjoy either.
That's life however - you only group with those of like-minds.
We are completely different in our ideas of "fun" so they are not loosing a potential game mate since I would never be playing with them from the beginning - if FFXIV did not include fast travel, I would never play the game, therefore it would be illogical to conclude that if fast travel were to be done away with, they would gain me as a game mate.