But you do enter those rooms.
Just not whenever you want.

In fact all 3 are major parts of the storyline that you visit on multiple occassions. And Kan-E doesn't even have a room, it's a clearing in the forest with a pond.
Also you get to enter Aymeric's private room in Ishgard a few times.

What are you looking for? More freedom in the game to go where you please? At least lore wise, it's fair to them to have their privacy. How would you like it if Ishgardian Knights burst down your inn room door in The Forgotten Knight just because you're an Au Ra and you're under suspicion for being a dragon heretic? Actually, don't answer that.