The swirls are only when you actually have beast gauge accumulated. So in combat, yeah. Near constant swirly aura. Running around town and such, no swirls at all.