A temporary band-aid suggestion until a more permanent solution can be found:

Make it so that people cannot send you party invites from a different zone unless they're in your FC, one of your linkshells, or on your friends list.