It kills the purpose of Materia.

With materia they can make it possible for gear to do the Minor things they did in FFXI. Add to stun from Shield bash, boost the effectiveness of evasion boosts etc (I mean they could add things like this, not that they are currently implemented)

With time materia could unlock endless play styles. Evasion or defense, more attack or longer status effect times, upfront damage or bleed damage. These are the small things players could someday use materia to define. Use to make their characters unique (and if these sorts of things are balanced right, they could all be legitimate strategies therefor leading to the rise of player preference and identity over standardization).

But if you add mid battle swapping, you create a optimum as opposed to many preferences. Suddenly we can have it all, and will be expected to have it all. No longer can a player choose what they think is important for their play style, because everyone else would already have chosen it for them.

I don't speak for everyone, but I would much rather define myself. I would much rather say what matters to me. It gives me a unique value, best for certain situations "That boss doesn't give us enough time to heal our party, go find that Monk with the crazy stuns, that would buy us enough time to heal".