No, no harm in doing so as long as you don't intentionally fail at things and slow down dungeons etc.