Personally I'd like it if it filled up instead filled down. So a 4 man party could still be in the bottom corner and the box would expand up when other people join rather than having a 3 or 4 man party half way up the screen.

The numbers are quite usefull for macros and stuff for example my tanks shirk is via macro /ac shirk <2> because that's always where the other tank in the party is.

I think the same for things like the duty list would be good or even make them flippable. My duty list typically sits on the left edge of my screen which can sometimes be an issue as the finger icon when you cycle active frames is off the left edge of the screen.