Go in undergeared and they'll still kick your ass. (I remember how frustrated I was that, after leveling up ROG/NIN as fast as possible in 2.4, I got murdered in the Castrum because my gear sucked. Good times!)
The "dungeons are too easy" complaint stems from the fact they're designed to be doable at the minimum iLv, when most players grossly exceed that on their first run.
If they add Castrum Oriens as a dungeon and if they set the minimum iLv high (say 235), people are going to gripe about how it's too hard and you'll get people dropping just because it shows up in their EX "roulette." Anyone who knows Halls of Reflection knows exactly what I'm talking about.