It's very often used as an insult by the non LGTB community so I think that's where the negative connotation comes from. Also depending on who you talk to the term 'gay' carries a sort of stigma.
(I feel the need to clarify I'm not trying to offend anyone by making this post, nor am I saying that all of the non LGTB community is negative like that, just making observations.)
