That's not true, even harcore players don't wan't the game to become too hard because they know all players don't have the time for that.
The biggest problem in this game imo is endgame content becoming braindead as time goes on. Some stormblood raids became easier and my best memories come always from difficulties i had to overcome (and i was a casual at the time now i'm midcore). It's more fun than just destroy what shows up and take the loot. I don't have any good memories doing e1 to e12 other than the 1srt time.
Even casuals can enjoy some good difficulty. A lot of them didnt have any problem when stormblood raid were harder etc