I liked the OP because it asks for harder stuff and I'm always up for that, even if I think the points it tries to make are kind of "disjointed" (my word of the day). Plus I'm not sure my definition of a hardcore enjoyable fight is anywhere near the same as the OP's (I enjoy complex mechanics challenging group coordination, which are resistant to attempts by players to trivialize them not simple gear checks/burn fights though a few of those are acceptable).

As an endgame aficionado I am only currently playing due to a series of comments made by Matsui and Yoshida hinting at very promising endgame activities in the future. I don't feel like finding them but they were part of the interviews they did and went something like this:

Yoshida: Yes we plan to introduce alliance content however we would like for that content to be more challenging and involving than simply slapping more HP, attack, and defense on current mobs to warrant an alliance of players attacking it so we are holding off until the mechanics in place to design such encounters are ready.

Matsui: We are taking a big risk introducing the current crop of endgame encounters while the battle system is not finalized.

From those I gather these are not even close to the difficulty we can expect in the future and are instead characterized by a few gimmicks which enable the dev's to simulate a challenge (which I think should be reassuring to people who struggle with Moogle/Ifrit as well as the issues they have with such gimmicks largely caused by lag may not be an issue for them in the future).

As for overland competition I am hoping the 2.0 version of primals with their free company unlock-able summons will fill that need nicely.