There is nothing about minions that should cause that large of a performance hit, even if you actually had 100 of them onscreen (which in reality is not very likely) as you get to a point where players are having more and more of the advanced gear, it is likely that drawing all the different gear is the problem.

Simply reduce your graphics settings until you are at a performance that meets your goals. Please note, most people assume that because they are at or above the recommended specs, that they should be able to play the game with every bell and whistle turned fully up. It is your responsibility to troubleshoot with video options until you get a frame rate you are comfortable with, even if that means making the game look like crap.