I haven't had this issue, I've been rather fortunate...

Brayflox HM- 1
Dzemael Darkhold- 6
Halatali HM- 2
Snowcloak- 6
Aurum Vale- 2
Hauke Manor HM- 3
Lost City- 5
Sastasha HM- 5
Wanderers Palace- 6
Copperbell Mines HM- 5
Hull Breaker Isle- 5
Qarn HM- 3

So far it's been fairly smooth sailing.