Agreed, there are far too many screens to back out of when you just want to get out of all of them. (I've noticed this particularly when you're looking at anything accessed from the menu. Journal and Config are the worst, in my opinion, as both can have up to 3 screens you have to back out of before you can exit the menu.)

I've also noticed that after you finally get to stop talking to a retainer/NPC, you're effectively locked for a good 3+ seconds after the end of the conversation, especially when you're interacting with your retainers in wards. It's very frustrating to be unable to move for no apparent reason after that interaction. Can this please be fixed?