Latency, yes. Loading times, no. I have a (good) SSD, so file loading times are not an issue. My GPU is powerful enough to not deep under 30fps most of the time, and at the moment this is happening in moogle, it runs at ~50ish. Moogle is the only fight where I'm having more than a second delay on models displaying.
The rest is communication between my client and the server -> latency. Also remember, SE said that slow NPC load is due to server architecture, not PC power. That was back at 1.02 or something.
I'm not blaming latency on me failing simply because I don't. I can't find a group to even consider doing Garuda, so all I did so far is killing the mini one.