Given what all you've said, I really wouldn't worry about it. If you do the sidequests as well (And there is lore attached to those as well), you likely won't have to worry about grinding for xp except for a small section around level 49... and if you've been doing your beast tribe quests once you unlocked them, probably not even then.