I know that Yoshi-p stated that the devs balance jobs based on the 'difficulty of playing that particular job and its rotation, as well as its support actions and their effects' during his apology for the P8S job balance, but I don't actually believe that to be the case in practice. What's more likely the case is that jobs are currently balanced in a reactive fashion to player feedback, with SE trying to cater to perceived popular demand.

Job 'difficulty' has no fixed goalposts, because anyone can claim to find anything difficult. It also comes out in the wash when you balance for an encounter. If a job is 'intrinsically' harder to play, you would expect players to put out less DPS on it prior to rebalancing. Do you really believe that a 50th percentile PCT is more skilled than a top 1% MCH? If you're balancing around 'difficulty' alone, you just need DPS parity. Match 50th percentile with 50th percentile, and top 1% with top 1%. That's balancing for difficulty.

A more accurate statement is that job balance is dependent on uptime and burst potential. Melee and ranged is a classic example of this. In the extreme case, if you create a boss with a moat around it that you can't cross, tanks and melee have 0% uptime and do essentially no damage under all conditions. In contrast, if you have a target dummy with an arena-sized hitbox, there's no advantage to playing at range. This dependence on fight design means that you either design all the fights to be the latter case (which is terrible design and very unsatisfying for players), or you just accept a degree of DPS discrepancy across various fights and try to mitigate it in other ways (i.e. downtime actions or role locks).

The same is true for consistent damage and burst. In the extreme case, if you have a fight with alternating downtime intermissions and burst windows, consistent damage jobs fall massively behind on damage. In contrast, if you have a long duration target dummy fight with 100% uptime, there's less of an advantage to burst. You can't balance both of these cases simultaneously. So you do have to have jobs with consistent damage output do more damage under target dummy conditions, because they're going to tend to fall behind in the actual thing. You can also try to mitigate this difference in other ways, like resource-gated (or more accurately, 'uptime-gated') burst and caps on damage variance (guaranteed Crit/DH).

I think the end goal should be to give the closest level of DPS parity within the DPS category for as many fights as possible. There will be a degree of discrepancy that is always present due to fight design, because if you balance jobs based around target dummy conditions, some jobs will be consistently behind. So you do have to uptitrate the DPS in those cases, such that they push ahead under some conditions and fall behind on others. You'll know you've hit the correct balance when the same job isn't always on top.

I think the 'damage' vs. 'utility' side is probably harder to reach a consensus on. The primary function of a DPS job is to do damage. So you do have an obligation to independently balance DPS with DPS, and utility with utility. I know that there are some Physical Ranged players that like the traditional 'support-focused' aspect of the role previously. But we're also at a point where Magical Ranged offers simultaneously more utility and more DPS than Physical Ranged, which is a failure on both counts. It's a cross-roads, but something needs to change, or you'll eventually find that players don't want to play Physical Ranged anymore.

I'll also say that I don't think that walking casts will result in parity for Physical Ranged. The goalposts around 'difficulty' will just shift, and you'll be told that you're doing less damage because standing casts are 'harder' than walking casts (even when those 'standing' casts are actually instant). Don't get me wrong, I think it's a fresh idea and I'd love to see it implemented on some current or future jobs (not necessarily even just Physical Ranged), but I wouldn't pin your hopes on it instigating a change.

To overcome this situation, SE needs to take a stand, throw out community biases, and say that they want to give equivalent value to the Physical Ranged subrole. I also think this will be a catch-22 as long as Physical Ranged is in a different subrole category from Magical Ranged, simply because there's no incentive for actual balance if you've got a guaranteed slot, and there's no way to accurately detect the drop in interest in the role with role locks present.