Fire deals 180 potency over a 2.5s cast, which is 72 per second. However, it can proc Firestarter 40% of the time, which, even when simply thrown in Astral rather than used to swap, is 260 over 2.5s. If you do a weighted average you see that Fire has an actual potency per second of 78.4.

Fire III deals 260 potency over a 3.5s cast, which is 74.3 per second.

This isn't even getting into the problem of Fire III consuming over twice the MP of Fire. If you're going to refresh Astral, you should always do it with Fire, not Fire III. There are also some issues with never using Blizzard III but they're less egregious than this mistake.