Buffs have an arrow pointing up. Debuffs point down. After a while playing you just learn what the different debuff icons look like so you know what they are anyway.