Downvotes are ripe for use as a harassment tactic. So, no.

What I'd prefer to see is a way to hide entire threads based on who the Original Poster is. If you've put them on your 'ignore list', the entire thread becomes invisible.