Annoying? Yes.
Life-threatening? Situational.
Easy to counter [losing your items]? Yes (by not using).
Needs changing? Not really.