I don't remember exactly where, but I believe in the dialogue they mention that area is dangerous at night. The reason for that is that undead monsters (The 5-star Dullahan and the 3-star Wraiths, iirc) spawn there at nighttime. Go during the daytime, it should be mostly empty.