I think the issue is being exacerbated by the discord servers popping up regarding the rebuild. People are organizing timetables of the upcoming fates on all servers across data centers (I'm looking at one right now). People aren't happening to chance upon a fate's time to fire off, they are planning in advance and camping to make sure they get in. So when you have, for instance, A at 7:00, B at 8:00, and C at 9:00, B is getting flooded with the horde from the discord (which filters out to CWLS and LS) to the next one to make sure they get in. So players on their own home worlds are getting locked out for long periods of time.
But you make a really good point though. It would be just as rude to force kick a Wanderer out right as it starts after they spent their time waiting.
Just my personal opinion: I think Wanderers should NOT be allowed inside instances of The Firmament with a FATE coming up, and five minutes before it is scheduled to begin, anyone can get in if there is room. Give the Home World players ample time to get in first. If you are a Wanderer in The Firmament and the progress bar completes, then you are given notice you have 10 minutes to leave the instance or you will be removed. Also, add an AFK timer to ALL players on ALL worlds of an AFK timer in The Firmament - just like in Eureka. There have been tons and tons of people hanging out in Firmament just because it is new and there is a crowd. People are standing around emoting afk for hours, etc - and they are preventing the people who are trying to participate from entering.