Yah most of the invisible walls likely exist not to keep you from jumping off high places but keep you from glitching and accessing areas of the map that aren't playable. The Castrum in Shroud is a good example. There probably isn't even any collision detection over there so you'd probably fall endlessly through the ground lol