It would be cool if you could simply fill in any gaps with NPCs yeah.
In the mean time, it sounds like you need to find a fourth like-minded person. Or Unsync the lower level dungeons.