Making it per class basis is a lot better honestly because there is no flexibility currently -

You mainly play as a DoW, therefore you're logically going to allot to STR, DEX and VIT (maybe INT if you care for after effect proc rate) for say 30-50 levels -- switch to DoM, you're horribly under skilled that no amount of point reallocation will fix - Physical Levels are only existent for the points awarded upon level up and can easily be applied to ranking up on the class and even allow stat growth based on your current class.

Generally gear is supposed to "Make up" for what is lacking, e.g +10 INT piece etc, which we sorely lack in terms of equipment but in surplus of elemental resistances which seems to do not so much anyway.