It's a pretty significant hit, around 20% or so for 1 rank and increasing more gradually after that. In only a few cases is it worth equipping higher rank gear more than 1 or 2 ranks before the optimal rank. Keep in mind the durability loss is much higher for items above your rank too, so even if the Iron Longsword gives an extra 5 to attack over the Iron Gladius at R30 (I'm making this up), it's probably not worth the extra repairs.