Speaking for only myself:
The main scenario narrative arcs alone carry some of the weight of my praises for the story, but more weight is carried by the consistency and depth.
To borrow a metaphor from Koji Fox, if you think of the story as if it's a cruise, many people are content to occasionally look over the side or out a window as the boat moves from Point A to Point B. However, if you grab your binoculars and scan the horizon, or disembark at every port to wander off into the wilds, you're not going to find a bunch of cardboard cutouts and painted curtains. (Not for an admirable distance, at least.)
A surprisingly large amount of the world is alive. Every character has their own identity. Every faction is working towards its own ends. When you look closely, you find organic motivations and behaviors, and characters (generally) act consistently with their identity and experience in context with their current situation.
Landmarks and dungeons bear hints of hidden histories that will never be spoon-fed to you. Data logs and side stories help fill in the gaps people find themselves unexpectedly fascinated by. Conflict isn't two horizontal lines meeting in the center, it's a bunch of diagonal ones awkwardly coming at one another as the world keeps on trucking around them. You can find bits of DNA from other events sneaking in if you're paying attention.
The more you know about the world, the more you appreciate how large and polished the story actually is (or, in some cases, how convincingly they make it look like it is, lol). If you invest that effort, there's a good chance you'll end up being one of the high-praisers. But if you don't feel like it, the cruise still ain't half bad.
EDIT: I feel compelled to throw in a random example to justify the above.
In the Main Scenario, you fight a Siren on the beaches of the Isles of Umbra that later becomes the dungeon boss for Pharos Sirius. The reason the locals are so freaked is that allegedly the sirens are supposed to all be dead - the pirate king Mistbeard killed them all in a great culling when he ruled the high seas. The half-cracked Lalafellin smithy Mimidoa claims to have been a part of this culling; he's been claiming this since 1.0, and it always seemed like one of his many tall tales. Suddenly, here you realize that it's obviously, comically true: his specialized earplugs help you fight the siren without being charmed.
Now, a different legend about Mistbeard claims that he once raided the wedding of Nanamo Ul Namo's parents, making off with a ton of gold and allegedly the (willing) virginity of the bride. That last part is probably an embellishment added as the story spread through ships and pubs, of course, but take a look at the map next time you're in Sastasha. If you look closely, the final room (the dock where ye ... weight your anchor) is called The Sultana's Lap. Once upon a time, it was one of Mistbeard's many treasure coves.
Eventually (Leviathan / Hullbreaker Island), you learn Mistbeard's identity, which adds a whole new layer onto his legend and connects to the legends of admiral Merlwyb and her father, as their pirate crew was a bitter rival of Mistbeard's. When you understand the relationship that the two now share, you then understand in a new way how Limsa Lominsa has evolved in becoming a legitimate nation state to ally against the Garlean Empire. And on and on it goes.
It spirals downwards and interconnects, from obvious ties to subtle hints. When you have all of that buried in a main narrative that itself is pretty good, people are willing to fawn over it a bit, lol. Not everyone, but enough to explain why you've read what you've read, even if you don't see what they saw.