To be honest are you shure its SE's fault? if so wouldnt other datacenters also suffer from that issue?
I've read on here that the US-canada backbone connection isnt the best leading to a lot of issues. Couldnt your problems be located there?
The fact that other games or general internet is fine means little since they use other pathways. The only thing you can be relatively shure of is that your own installation or local provider isnt at fault.

I know at times connection was really bad for me when chaos datacenter was still in canada but all issues seem to have been resolved by the migration to europe.