Playing with no Job is for soloing, playing with a Job is beneficial to party-play. Not having to grind that Job is actually better for the casual player to hop in parties without having to go through another 50 levels worth of grinding just to catch up to other people.

Yes people may consider you gimped if you are not using Jobs after you have unlocked them, specifically to party play. Maybe some mobs will be too tough to just go into fighting carelessly. All jobs are pretty much just specific roles, nothing more, nothing less.