False. You have considerations to make. There are also priorities. Then there's the fact that if movement is involved during the fight, you have to keep track of that plus your position relative to the mob. Have to know when to pop cooldowns and when to hold back. If there's a burn phase you have to keep an eye on what others are doing and listen for your raid leader to call for burn.
That's not mentioning encounter mechanics, boss moves and other things, especially if you have offheals or buffs to keep up. It's not as simple as just standing there and pushing buttons in a sequence.
Would you rather have a system like XI's summoner where your summons and summon abilities be useless 9 times out of 10 outside of a handful of abilities. No one focused on Titan or Ifrit; not when everyone knew Garuda had highest physical damage with Predator Claws, and Shiva was not far behind. Guess which two were the most used summons, with Carby slightly edging them out because you could gear yourself to nullify perpetuation costs for Carby.
Bringing out summons only to buff and dismiss is a large part of why the job suffered as much as it did. The fact that out of the 12(?) summons in the game only two were remotely useful outside of buffs was also a glaring case of having too many elements watering down the system. So in that regard, I agree with the way the devs are handling summoner. Not to mention, chances are they are currently sticking with 4 avatars for now due to the number of primals introduced in storyline. They'd be tipping their hand if they had told us that SMN will have Ifrit, Garuda, Titan, Odin, Leviathan, Phoenix, the Magus Sisters, Ark, and Maduin.