It just ruins the "into the game" feel.. thats all.. IRL you are also not blocked by invisible walls...
And I'm not asking for them to make the game uber realistic... but stuff like this is basic... and should be unacceptable.