I realized with editing my post I can actually show the math, so here goes.
Code:
3 mobs 4 mobs 5 mobs 6 mobs 7 mobs 8 mobs
F3 168 168 168 168 168 168
F2 540 720 900 1080 1260 1440
F2 540 720 900 1080 1260 1440
F2 540 720 900 1080 1260 1440
F2 540 720 900 1080 1260 1440
B3 168 168 168 168 168 168
Freeze 300 400 500 600 700 800
Total 2796 3616 4436 5256 6076 6896
18s DPS 155 201 246 292 338 383
This beats the F2 x 2 -> Flare rotation but needs full MP. That means we actually need to compare it to this:
F3 -> F2 x3 -> Flare -> Transpose
Code:
3 mobs 4 mobs 5 mobs 6 mobs 7 mobs 8 mobs
F3 168 168 168 168 168 168
F2 540 720 900 1080 1260 1440
F2 540 720 900 1080 1260 1440
F2 540 720 900 1080 1260 1440
Flare 1264 1591 1872 2106 2340 2574
Total 3052 3919 4740 5514 6288 7062
18s DPS 170 218 263 306 349 392
This is about the same time frame assuming roughly 3.1s for F3, 2.6s for F2/Freeze, 3.5s for Flare, and the worst case 3s for Transpose's server tick, so I just used the 18s total estimate. Actually the Flare rotation is slightly faster, but it probably evens out considering you will cast F3 in UI3 for the Freeze one.
I think the theory is sound because thinking about it, you gain 234 potency on Flare for each mob past the 5th, versus 180 for F2 and 100 for Freeze, which should eventually cancel out the big initial Flare bonus, but it certainly needs more than 8 mobs. I'd do the math to figure it out myself but I'm real bad at formatting these tables. For the rotation AFTER you Flare, yes the Freeze one beats out the Flare rotation, but you'd have to Blizzard 3 from UI1 and wait for a server tick, which seems suboptimal. You also have to consider, as more enemies die, Freeze gets worse, so if you expect anything to die within 18s and drop the total amount of enemies below the magic number, you should do Flare anyways.
But at a certain point, Freeze
should actually be better! It just needs a lot of requirements that aren't likely to be met (enemy #, UI3, enemy durability).