Make Paladins heal, then. Or else you'll still have the same problem if you get a group with two Paladins.
Using Sword Oath to attack as Paladin doesn't work, as you'll steal aggro from the main tank.