Amen. This is my problem with the game. In order to progress, everything is gated and capped in such a way that it really just feels like a chore to play and do. Part of it I realize is the very limited content in which to do so, but in a larger more general sense, the tome system as currently designed really doesn't not inspire fun. The fun I have had with the end game comes from friends in a few LS and the FC. Not the content. I even unsubbed once, even though I'm relatively new to the game (compared to beta and original ARR players) and I don't know if I can stomach a new tome and a repetition of this design. We'll see. I'm working away at it, but, emphasis on work, as I agree with OP that it really doesn't feel like play most of the time.
This would help. Not a complete fix, but I might actually start playing other jobs again. As it stands now, having returned from a hiatus, I have retired everything but my main. It's enough to try to get a full tome set and upgrade it for one job, get the relic for one job, and do some crafting and gathering to pay for what the job needs. A second job can be decently equipped as a side result of VA weekly runs for farthings, but not so much if you wish to have multiple jobs in tome gear.
As for the RNG thing, I know most people hate it and condemn it, but I'm with the OP on opinion about it. I know the model is not likely to change for FFXIV, so my example is merely rhetorical, but I loved Phantasy Star Online and its sequels. RNG goodness, yes, but the thing that made me play the game for thousands of hours between editions (and I'd still be playing if only it would get localized again) is that even with its limited content it was always fun to play through things you played a million times because you might get that really rare thing you've been wanting a long time, you might see a rare enemy spawn, but quite often you're going to get some things you already have but it better versions - and - you could still play the game just fine not having max equipment. I think the one lesson FFXIV could learn is that tomes aren't really great motivational rewards for replaying the same content a million times. There needs to be an edge of excitement present each time you run it. PSO had it. FFXIV does not.
And this is a fantastic reference. I remember reading that when preparing to DM my first campaign. The reason I love FFXIV is that they do some fights well and they are fun for a little while, but mostly, it's the people I meet and become friends with - and running the same old things with them. That's what keeps me playing. Granted, I'm in a sense creating my own fun through the presence of the people I wish to play with. I think SE could perhaps find some better solutions. Sometimes it feels like even though they design the content, they don't actually play it for months on end non stop like we do, else I think perhaps they'd design it a bit differently.
Finally I want to add: you only criticize what you love. If you don't love it, why bother? Just leave it behind. Things I might say about the game don't mean I am negative about it per se. It just means I can clearly see things I don't like about it and wish would be improved. For whatever reason the OP struck a chord with some of my feelings about the game and why I unsubbed in the first place, although I am indeed back.