I'm not seeing any disadvantages either other than someone not wanting to speak to an NPC while in town. Also thinking you'll be penalized just because you're not getting a tiny boost. When you start LOSING seals for not having it, then we're getting penalized.
