Simple solution to mount speed killing diversity: have the speed tied to your character stats that you can either upgrade through quest and/or whatever.