The "casuals" you're referring to don't even need tomestone gear unless they plan to do the content people like to label as "hardcore". So then these "casuals" then get the gear needed to meet the ilvl requirements to jump into the "hardcore" content because they want the shiny mounts, sparkly trial weapons, etc. They spend 95% of the time on the floor because they were allowed to stand in AoE markers pressing 2-3 buttons most of the time due to current dungeon design and easy mode settings in the MSQ.
As for it being a grind. Yeah anything feels like a grind when its boring. Dumbing it down to run until you hit wall then mash AoE until duty is finished just makes the content boring right from the start even on the first run because it's already boring by initial design. Instead make the dungeons more difficult and involved. So the people running them repeatedly are at least getting practice dealing with mechanics and variables they will see in that harder content and these "casuals" become less likely to be a drag or hinderance to people when they decide to jump into other things.
You could jump into EX trials in DF in ARR and have a fairly good chance at clearing it. There was a reliable chance of clearing HW's 2 initial EX trials with random people before they nerfed the 8 player duty at the start of the HW MSQ after players not doing the mechanics complained non stop for weeks about being "gated" then the rate of failure started increasing. The initial 2 trials in Stormblood were the same. They were cleared with random people reliably until they nerfed Shinryu normal after people again complained for weeks that they were being "gated" out of content. I see all content as accessible to everyone so long as they are on a version of the game that allows access to it. However doing said content should involve the player actually learning how to effectively play the game. The dumbed down 4 player duty design and nerfs to various duties and the MSQ is nothing but an act of appeasing lazy players and enabling their behavior. Something that I feel should not be done because it does nothing but damage the player base. It's the game equivalent of giving a child screaming and throwing a tantrum in a store, toys and candy because you don't want to listen to it. And when others lacking morals, ethical standards, and integrity see rewards being given out for essentially being a little prick they start being a little prick as well.