I'm all for a duel system of some sort. Maybe if you don't want it to be publicly fighting everywhere, (I can see some issues with accidentally accepting a duel challenge in areas filled with monsters), it could make use of the actual arena in Ul'Dah in some way or form. Such as maybe have it where you can either enter the ring via the DF solo to get a "random match", or as a party of two to have a "private match"......
If such was implemented, I would also be all for the idea of allowing spectators via talking to an attendant of some sort in Ul'Dah at the arena itself. It could be a payed attraction to do when you're bored, (maybe just a couple hundred gil), and have a tournament system implemented as well. In addition to the bragging rights, the victor could receive a set gil amount plus 50% of the admission fees from any spectators that watch their match. If implemented properly, it could give us both a form of entertainment to the world, and a fun way to individually test our skills against each other if we so choose......
EDIT: After going back and reviewing some of the previous posts in this thread, I would like to point out that a duel system is no where near the same as open PvP. I don't really see why all the PvP haters out there seem to think that allowing organized duels of some variety is the same as allowing PKing and harassment in the open world? I personally hate open world PvP because said harassment is so easy, but dueling is in and of itself something different entirely. It's an agreed upon challenge between two combatants, (be they friends, rivals, or two angry people who think that each other is bad at the game), to test their skills against one another, and nothing more. Claiming it would cause any more drama than what already exists is baseless. People will still harass each other claiming that they're better than the other regardless, this is why the "report player" option exists to begin with. Creating a dueling system simply allows two players who feel the same to put their gil where their mouth is in such situations. Harassing players will get the same treatment they currently get......