Here are my thoughts (take them with a massive grain of salt):
There's something to be said about competition. HNMs with 72hr repops do provide sufficient competition to players. But there's little strategy here. You know exactly when it will pop. However, setting a range on the pop timer will not help either, nor will making it a lottery spawn. What really needs to be present in a HNM that drops really good gear is the following:
Strategy. It shouldn't be a one size, zerg all monster. Some serious thinking needs to be done on the devs part to make the monster engage the players in battle. It should be legitimately hard. The gear is good, therefore, the players have to play well to get them.
Absolutely NO randomness. Nobody likes random. But drop rates exist for a reason: Limit the number of existing pieces. That makes sense, that's the competition. So drop rate should be random. (You're welcome thieves...) But access to the monster should not be random. Progress bars are great motivators. If you are constantly making progress to get to fight a monster, you get to see that. Randomness in invisible and it annoys the life out of players. Suggestion: The pop shouldn't go away if you lose to the monster, perhaps some farming needs to be redone, but failure is always an option. We can only learn from failure, so don't punish us for learning.
Good drops. Man, they have to be good. Tons of work should go into getting the monster to pop. Legitimate work. It's gotta be hard for it to be an HNM. Then, the monster needs to be tough. You have to really have great players with great gear and a great strategy (something unique). So it better drop great gear. Grand difficulty means grand drops. They don't even have to be gear drops. It could drop 0-3 items that you turn in for gear or something.
Play style adjustments. Legion is interesting in that there are two modes: 18 and 36. Each mode has its own benefits. HNMs should be accessible to single-party groups. That could mean allowing the party to pop another HNM that's less difficult and drops less items or lower-quality items, etc. Every player has his own preference. Play on that.
Interesting story. Not a requirement, but story really beefs up the entertainment-factor (this is a game after all). I really enjoy beating the tar out of Shinryu because of story. I'll do it for fun. I don't even need to proc. I enjoy that kind of stuff. I know a lot of other players do too.
I think that what I'm trying to say is this: There are a lot of systems in this game that do one thing great. I'd like to see a new system, perhaps new HNMs, that really hit the nail on the head, here. I'm predicting that's where the real end-game content will end up. To the devs: Please make it fun, rewarding, difficult and yet accessible to most players.