Attribute points are separate for each class. The only class you have to think about will be Arcanist as it can become either a Summoner (INT-based) or Scholar (MND-based) but the points won't hurt you too much in either case.