Occlusion Culling Perfomance Issues/Drop--Comprehensive

[ReiqMagnus]

Just out of curiosity, I tested the affects of Occlusion Culling on memory usage (system, private, working, commit, GPU) and needless to say, on/off seems to have no effect on memory usage, so the game is clearly not releasing the occluded data from memory; here are some screen shots--the highlighted row in task manager is ffxiv.exe. (ignore the crazy fluctuations in Afterburner, they were caused by tabbing around)

Enabled (OC on)
http://i.imgur.com/xUg8ntu.jpg

Disabled (OC off)
http://i.imgur.com/oBIMXCe.jpg

Granted, I am not a computer science major, but I am quite sure this proves it is not a memory-related issue.

[Squa11_Leonhart]

afterburner and task manager only provide a small portion of whats actually going on memory wise

use process explorer instead.

[ReiqMagnus]

Since you seem to be insisting it is a problem on my end (even though my CPU is still better than 85%+ of gamers) what memory values do you want me to look at? There is no significant change in GPU system, dedicated, and committed bytes when OC is on/off or when the camera is moving around, nor is there a change in working or private bytes (other than constant small fluctuations for both system and GPU memory).

If these problems are somehow related to CPU or RAM limitations on my end, you can be rest assured over 80% of the player base is experiencing the same problem, whether they realize it or not; whereas they either have low-end systems and lag all the time anyway (very, very common in my experience), complain about lag but have no specifics on its nature (all too common), or attribute the lag to "normal" since it is a highly populated MMO (again, all too common).

[Syx]

Actually, your memory usage dropped by about 15-25 MB. Which seems on par with activating OC. My theory is FFXI poorly uses the IDirect3DQuery9 command for calculating meshes to cull. When you move the camera within the game, it has to recalculate all of the meshes, identify the polygons that can be discarded, and render the scene. That recalculation adds a few milliseconds to the rendering of the scene, at which point the calculations complete and the benefits kick in. So what you're seeing is this backface culling engine at work sacrificing 8-10ms to recalculate all faces of every model's mesh that are not visible on your screen. Yes, this includes characters and objects that were not visible on the screen since the last time the algorithm was used.

The problem is, IDirect3DQuery9 is not a terribly efficient method of culling on its own. Sloppy usage of this query results in lots of draw call overhead and noticeable latency while the GPU waits for the results of the queries. There are algorithms for reusing query data, but there's no way of us knowing if SE uses them without some reverse engineering or a drinking with chatty dev. Anywho, I tested your findings on my system and I see the same drop in GPU (GTX 660 SLI) usage, but I also see a shift in CPU (i5-3570k @4.2GHz) usage. All four cores seem to change their utilization. What I did was slowly pan the camera around the market area in Ul'dah while watching my G15 keyboard's LCD, I saw a 15-30% drop in GPU usage while panning and a 20% spike in CPU usage on one core, with a more moderate 10% increase in the remaining three cores. My fps would dip from 42-45 (22-24ms) down to 36-38 (26-28 ms). Not as drastic as your system, but still a variance of about 2-6 ms in draw time. With culling disabled, my fps remained a constant 36-38 fps during the same panning action as well as stationary. Take into consideration my framebuffer usage is smaller due to running the game at 1080p. Might be time to upgrade your CPU! Hope this information helps.

[Squa11_Leonhart]

Actually, your memory usage dropped by about 15-25 MB. Which seems on par with activating OC. My theory is FFXI poorly uses the IDirect3DQuery9 command for calculating meshes to cull. When you move the camera within the game, it has to recalculate all of the meshes, identify the polygons that can be discarded, and render the scene. That recalculation adds a few milliseconds to the rendering of the scene, at which point the calculations complete and the benefits kick in. So what you're seeing is this backface culling engine at work sacrificing 8-10ms to recalculate all faces of every model's mesh that are not visible on your screen. Yes, this includes characters and objects that were not visible on the screen since the last time the algorithm was used.

Yep, i can confirm this behavior having seen the world and pc/npcs vanish and pop-in simply by moving in and out from behind someones chocobo in first person view.

I'm hoping the D3D11 client implements multi-threaded display lists and tesselation, the game is rendering an obscene amount of polygons for distant objects for which tesselated sub division should improve gpu usage (lowering it) for the same level of detail.

[ReiqMagnus]

Not as drastic as your system, but still a variance of about 2-6 ms in draw time. With culling disabled, my fps remained a constant 36-38 fps during the same panning action as well as stationary. Take into consideration my framebuffer usage is smaller due to running the game at 1080p. Might be time to upgrade your CPU! Hope this information helps.

Indeed, I am aware of my aging proc; Intel's genius decision of ceramic thermal paste in place of solder to attach the IHS to the cpu die, resulting in anywhere from 5c-10c higher temps accross the board has made me reluctant to do so, since I am not exactly a fan of the "de-lidding" concept as the modders refer to it as.

But in any case, going by the latest steam hardware survey, if this issue is a combination of CPU limitations and their admittedly insane draw distance which users cannot change--for example, I can be in the Alchemists guild and still get massive performance drops if I look in the direction of the marketboards--the performance issues related to it are overwhelmingly widespread; not even 15% of people in the survey would be using what could be considered a higher-end ivy/sandy or haswell proc, and since this game caps out at 4 cores/threads, AMD users are going to be in even dire straits.

[Syx]

Did I say FFXI? Wow. Flashbacks of the same issues experienced in an eleven year old game. Thanks Squeenix.

In response to Intel's zany thermal dissipation designs, all-in-one water cooling devices for the CPU are now affordable, efficient, and function well enough to counterbalance the increase in temperatures caused by poor construction by Intel. On my 3570k I never get above 65 Celsius at full load using a Tuniq Tower 120 Extreme air cooler with the fan at the highest. http://www.frostytech.com is a great resource for finding a CPU cooler that will fit your needs.

[ReiqMagnus]

I usually tend to avoid frostytech as their 150w load is quite unrealistic; my 4GHz i7-930 at 1.3v w/ HT enabled (I use SVP, kind of the only reason I use it) far exceeds 150w--stock TDP is 130w at 2.8GHz with 1.25v vcore, 1.35 QPI. I use a Thermalright TRUE Spirit 140 with two fans which still ranks up there with the Archon and Silver arrow on high clocks. Once you start hitting high clocks, the thermals increase two-fold--increased frequency and greatly increased voltage requirements. Higher frequency = lower electrical resistance = higher current = higher wattage/heat generation.

From what I have read and seen from "de-lidded" comparisons, it doesn't matter how amazing the cooler is if it is passive (whether it be air or liquid) since the ceramic TIM acts as a huge bottleneck transferring energy from the CPU die to the IHS to the heat-sink, in that order. At lower clock speeds it is not too much of an issue, but the temps really start to skyrocket around the 4.4GHz+ area--people getting 80c or higher even on top-end closed loop systems. Personally, I would want to shoot for a minimum of 4.5GHz since it seems to be around the sweet spot/plateau for 4770k procs, just like 4.0/4.2GHz is for 920/930 procs.

Unrelated to to your reply, I have noticed the performance issues have gotten a lot worse recently, around the same time they released the "fix" for draw distances related to coils. This could be related to that, or just the increase in server load/population over the last month.

[AeryM]

[...]and since this game caps out at 4 cores/threads, AMD users are going to be in even dire straits.

Tell me about it. I run an FX CPU @4ghz. Never had any issues with MMOs in the past - runs GW2 like a charm on max settings for example.
This bug is seriously ruining my gameplay experience, it actually forced me to take a break from the game, eventhough i love it.
In Uldah for example, while looking at the adventurers guild without moving the camera, i get 60fps stable (synched). As soon as i move around my camera it drops to 25ish fps, sometimes 30 depending on how crowded the zone is.
Here's hoping to the DX11 version. But i'd love to see it fixed before that...

[ReiqMagnus]

In Uldah for example, while looking at the adventurers guild without moving the camera, i get 60fps stable (synched). As soon as i move around my camera it drops to 25ish fps, sometimes 30 depending on how crowded the zone is.
Here's hoping to the DX11 version. But i'd love to see it fixed before that...

Same issue, same area/direction. The "hot spots" seem to be with character model heavy areas--player characters only I think, since in a lot of areas, such as instances and around The Waking Sand, both of which have a lot of relatively static NPCs a boat load of geometry, don't seem to have the problem.