I will just add my 2 cents here but
You are skipping the entire point of those sidequests. Those 10 boxes of text you skipped? They're the entire reason to do the quest, the exposition you get, the knowledge they give you about the game world.
Other than that, they really shouldn't be at all a big focus, yes. And hell I daresay that taking my first dude up to 50 would have been hell without the XP and item rewards from those.