Why not take it a step further and do this to make the buff bar cleaner:



Display number of stacks of the same effect from all sources (top in the image)
Display longest duration of the same effect (bottom in the image)

Keep personal buffs left-aligned and possibly larger, as per OP suggestion.