Yes that's just how the friends list work. It takes both parties to set up the friends link, but if one person removes it, it doesn't automatically remove them from the other person's friends list. That also means they don't need to confirm when you readd them because as far as they're concerned they already gave you permission/you're still on their list.

Because you are still the same character, when you moved server your name would have changed to "(Deleted)" on other peoples friends list/linkshells (because it doesn't handle cross server friends lists), but when you moved back to the server it would change that (Deleted) back to whatever your current name is (as your character is still the same).

Unfortunately there's no way to check if someone still has you on their friends list, but if you're getting harassment problems, just report them to the Square Enix anyway.