Another way i was thinking they could implement a nice HNM system would be introducing the element used in FFXII to hunt NM's or Marks as it was called there. You could introduce the Hunting Clan and reward NM hunting with achievements and other goodies while making some lore that this clan came together as a need to Hunt NM's causing havoc post Umbral Era. These could be done solo, duo, trio, etc. so small groups are only needed for them.

Also for Free Companies you can have the Elite Marks and make it a challenge for bigger groups. These could be introduced as a way to eliminate HNM's that aren't causing trouble yet but will be a hazard for the people of Eorzea if left unchecked.

Maybe have dragon sightings and other phenomena occur (credits to Hirmu) when these HNM's are in the vicinity, so players can have an idea when they are about to pop. I think that would add to a more interactive, rich and alive world experience without endangering people doing other activities in areas where HNM will manifest as they will not pop without a Free Company luring them to combat.

The system could be Tier based having Free Companies clear through tiers and always having the option to do previous ones without having to clear everything again from scratch. As for how many times you can do them: Could make a timer for it per Free Company or could make it so that certain HNM's only have a window of hours before the Elite Mark is removed from the Clan Bulletin Board. No need to claim bulletin board either for the mark as could have any Free Companies take a bite at it while the Mark is up for the taking in that window of time.

For the battles to be exciting and interactive i think keeping people moving around would be the way to go as the ARR removed combat lock. I.g. have a dragon cue he is going to use a move when he retreat its wings, when it lifts his head above usual that sort of stuff. Of course that doesn't have to be done for every move but for the ones you know you should seriously move or you'll be in danger.

As for rewards obviously some nice gear drops, achievements, maybe some sort of credit for the company as a whole (and add some something cool to display at the Free company housing to buy with those credits) and personal credits as well.