After doing this a couple more times I think they made the story thing triggers a bit too hard to work out without an obscene number of runs or turning to a guide.
Left Path
#1 Room with Green + Yellow Sacks, take the GREEN SACK in the NE corner on top of the rubble, and the YELLOW SACK in the SE corner on the very small hill of dirt, after clearing the enemies after teleporting, there is a door to the west behind the rubble that you can open before moving ahead. This door won't have a name or be interactable until Nanamo walks up to the gate to leave, do not walk up to Nanamo immediately if you want to open this door first.
#2 Same as #1, but do not open the door
#3 Primary Path, use two of the wrong sacks (don't activate the teleporter) and trigger the puff circles after the door opens
#4 Primary Path, use two of the wrong sacks (don't activate the teleporter) and don't trigger the puff circles after the door opens
That kinda stuff seems really cool on paper but trying to work it out without outside help just seems extremely tedious but at the same time just turning to a guide seems boring.