Okay I haven't read a lot of the thread, but I remember talking about similar effects in another thread, and I have my own proposition that might make the whole Reflect difficulties a bit easier to handle similar to the one from the other thread:
1) Reflect would be a single-target cast on any individual, can be made AoE by Accession.
2) Reflect would counter up to X number of spells, where X is based on your Enhancing Skill (meaning that RDM, with higher Enhancing, would rock this, but WHM and SCH could still get access and use it competently). Maybe like 1~3 spells to prevent it from being OP? Or more, but would require a longer recast time because monsters only cast spells so often unless under Chainspell/Manafont.
3) For limitations, reflecting spells would be limited to player-only spells to avoid stuff like Death and certain -gas from being countered. This would provide some aspect of balance.
4) For AoE spells, the original target is the only Reflect that will activate; if a party with Reflect on is hit by an AoE spell, the original target will use up one "use" of Reflect (see #2) while the spell is negated for the rest of the party. An AoE spell reflected will hit the target that cast it and any eligible mobs around it (example: during Voidwatch, you won't hit non-VW mobs nearby).
5) For damage/effectiveness of reflected spells, I'd say make it based on the player's stats, if that's not too complicated. Simply put that would prevent this from becoming overpowered while staying a viable defensive strategy. A melee with mediocre magical stats wouldn't reflect much damage, but would be protected from spells to an extent, while a mage who might get hit in the backline or even in front (the RDMs that do melee, or a WHM or SCH running in to AoE a buff like Protectra or Stoneskin-ga) can reflect it with more potency.
And to cover the issues that were given:
Reflecting Spells Cast On You
I suggest that player-originating spells ignore Reflect, simply because we normally cannot nuke/enfeeble/etc each other during battle. However, we wouldn't want Cures, etc to be bounced back at the user when a melee is nearly dead from physical attacks, because that would make this a very weak spell to use unless you have a tank who can turtle up enough to take hits until Reflect wears/is canceled.
Random reflection and AoE Spell reflection
Don't make reflection random, then. Have a reflected spell hit the target that cast it. Single-target spells will only affect the one who cast, but AoE spells will hit all eligible targets within range of the original caster (See #4, above). This includes debuffs as well as elemental spells, and with being limited to player-only spells, still requires that the mages be on their toes to handle certain situations (-ga debuffs, etc).
Effect animation
For this I'd recommend a simple shimmer effect (if possible) on the character who is being targeted, and the actual spell animation removed from the players. Since it would only be "targeting" one character with AoE damage hitting those around him, only the targeted character would need this effect, reducing lag on the server. Follow this by the spell animation being cast on the original caster, and in the chatlog something like:
"Monster A casts Fire IV."
"Reflect! Monster A takes X points of damage."
Stats
I can see the issue with this, hence why I made a limiter as above (see #2), which along with a decently long recast time (I'd say... 1:00~1:30, perhaps? I don't recall off the top of my head the average time difference between monsters casting spells, but it should be within 3~5 of their spells to be balanced, I think) would probably balance out this spell enough to make it potentially effective if used properly without making it overpowered.
And to mention stats, the damage of the reflected spell, as I suggested above in #5, should be based on the target that is reflecting the spell. If your character has low INT/elemental skill, your reflection of a nuke isn't going to be very potent, but you're still protected from it, giving you a defensive option without overpowering the damage on it. Mages, on the other hand, could make more potent use of the reflected spells, but would put themselves in danger if they're on the frontline with melee so it's more of a protection against backline nukes.