Hmm... what if the fee wasn't gil? Like the way to enter could be a Key Item that you have to do a pre-req quest for and then to actually enter you'd pay with... who knows. Maybe Guild Marks? Most of us have quite a bit of them and with the leve system in place it would stop you from being able to level strictly off this coliseum but it would also help prevent (to an extent) the whole RMT/bot thing. I guess it is kind of asking for trouble with bots if they could enter with gil, kill X mobs in Y time and repeat it without penalty.

Maybe it could be timed. What if it was like
1.) Entrance Fee
2.) Defeat Wave 1-5
3.) Mini Boss
4.) Defeat Wave 6-10
5.) Defeat Boss

And then you had to wait an ingame day or even possibly a rl day before entering again? It'd be cool to have something than the grind. I know we have quests but things like this and the whole Ul'Dah casino concept would be pretty cool.