The Saddlebag isn't the best thing to use as you have to keep removing stuff from it that you want to use, much like a retainer. One easy solution is to just make a new seperate tab in the Armoury Chest which is only for fishing tackle/bait. That way it's easily accessible wherever you are when fishing, but whether such a thing could be implemented is always a much different story.