The issue is tuning, how hard do you tune a fight, they chose 8 man fights because it allows a fair amount of balance between difficulty and numbers, if you was to have a 24 man raid on the difficulty scale of coil you have something that would be almost impossible to do for an even bigger majority than there already is the only similarity I could come across would be Dynamis back in the early days of FFXI where you need a highly co-ordinated large group of people otherwise you are going to wipe on almost every pull.
With 4 people you have to tune the fights even harder than 8 man (in relativity to skill/gear per player) to make a challenge, getting 4 completely co-ordinated people is much simpler than 8 and you would have to tune 24 down to a lower scale of 8 to make it fairer.
Take Titan EX for example, almost everything is 1 hit ko and there are reasonably sized areas to stand in to dodge. To scale it down for 4 people but keep difficulty intact the arena would be made even smaller and the areas to dodge would need to be even less which would require even more co-ordination to make sure nobody gets stuck in an area where it is impossible to dodge, anyone even one foot out of place is off the edge and its a wipe, originally in 8 man when it was first released that essentially meant you wasn't likely to complete the DPS checks but it was still possible and nowadays you can steamroll it even without echo and a few people falling off.
Now go the other way make Titan EX 24 man raid, firstly you would likely to be laggier due to the extra people on screen causing many people to get knocked off immediately secondly you would have to have 3x the amount of gaol's, adds and you would also have 3x as many plumes scattered around unless the entire 24 man raid stacks behind him. If you insisted on the difficulty remaining linear losing 3 people out of the 24 would start making it difficult to clear which at this point is highly likely to happen. I don't think I've seen a clear on WoD without losing a handful of people on every boss.
They originally had the Labyrinth of the Ancients tuned to a difficulty not far behind coil but when they tested it they found it too hard to deem worthy of release and then delayed it so they could nerf it to a stage they felt was fair. I think I remember Yoshi mentioning that the majority of the raid was similar to the Atomos battle and if anyone here remembers that around the time it was introduced it was a nightmare.
Would I like to see much more difficult 4 and 24 man raids, yes most definitely. But it poses a very difficult challenge for the dev's to maintain balance on those raids for the less hardcore since once you start making everything as hard as coil then you are likely to lose the majority of the playerbase.
edit: To Tiva below Yoshi always said the 24 man raid would be a step below coil and it was originally intended to be released in the patches before/simultaneously to allow people to gear higher before entering coil in later patches. It was never intended to be anywhere near similar in difficulty hence the delay to its release and nerfing to a stage that was reasonable
From live letter 8, dated 8/8/2013 (Pre open beta)
Q19: Please tell us what kind of bosses and mechanics are waiting for us in the Great Labyrinth of Bahamut and the Crystal Tower. Will primal equipment be necessary?
A19: As there are most likely newcomers watching, let me explain simply to start off. The Great Labyrinth of Bahamut will be the hardest content at the start of ARR for a while. It will contain a lot of things such as what happened after the Seventh Umbral event. While I can’t really say too much about the mechanics, there will bosses that can only be defeated by performing certain actions and that have different phases. Without having a party that can deal the proper amount of damage, you won’t be able to move past certain phases. Additionally, there will be shared damage type of attacks that you will need to have 4 people get hit by, as well as have a certain character act as a channeler for the damage. There will be a lot of different elements present.
The Crystal Tower is content that can be undertaken by a 24 person party. I don’t think that difficulty is really that hard, and we’ve adjusted it so it’s something that can be enjoyed by everyone.

Reply With Quote

