My ultimate issue isn't with homages or references - little bits here and there, like Cid being an (airship) engineer or the presence of common creatures such as moogles and chocobos doesn't bother me. Bosses and the like can get annoying (did they really have to make the endboss of the Burn the Mist Dragon?), but if it's just little tidbits not deeply ingrained into another world's mythos I don't mind too much.
When it's a cut and paste job of entire locales... that's when I start to get irritated. The Crystal Tower was a bit irksome, but they tied it into XIV's world well enough. The next big one was the Warring Triad, but their seemingly disparate origins made it tolerable. Then we get to Stormblood and the raid series is Omega... literally simulating major bosses and villains from previous Final Fantasy games (Deltascape's enemies supposedly come from Garlean legend; Sigmascape doesn't even bother with that thin excuse) with only two exceptions (Midgardsormr and Omega-M/-F). The Alliance Raid series is almost literally a cut and paste job of Ivalician locales and characters.
It's when it feels like Hydaelyn is being adapted to fit in the reference rather than vice versa that it gets... bothersome. "Return to Ivalice" is just the most egregious about it, in my opinion.