Currently, no, there isn't a way. If a quest is accepted and in your Journal and there's space on the tracker, then it will be on the tracker. (The closest we have so far is the ability to indicate which five should show when we have more than five. But we can't decrease the number shown.)

Whether they'll fix that, well, I haven't heard any indication they will, but I sure hope so. I agree, it's annoying as it is. I'd like to keep ones I don't plan on doing for a while off of it as well.